))
Ep2. Data Engineering at Uber and Lyft
Manage episode 283145520 series 2858756
Content provided by Thomas Wang. All podcast content including episodes, graphics, and podcast descriptions are uploaded and provided directly by Thomas Wang or their podcast platform partner. If you believe someone is using your copyrighted work without your permission, you can follow the process outlined here https://player.fm/legal.
这一期节目我们和嘉宾泉来聊一聊Data Engineering，在Uber和Lyft的应用和工作体验，对于行业发展的反思与展望，以及对于入行Data Engineering的建议。只要你对Large Scale Data Engineering感兴趣，本期节目都会对你有帮助 :)
Timestamps:
- 00:00:00 Intro
- 00:00:40 Data Infrastructure vs Data Engineer
- 00:02:45 Data Engineering 领域介绍
- 00:09:00 Pipeline权限设置
- 00:11:00 Pipeline在Uber的使用场景
- 00:11:45 如何追踪Data Owner
- 00:11:52 如何保护用户数据隐私
- 00:17:16 Data Infra转到Data Engineer的日常工作
- 00:21:44 Data Quality Tier
- 00:32:52 Uber vs Lyft 数据量级和迭代的区别
- 00:35:46 COVID影响
- 00:38:40 对颠覆性产业和商业模式的反思
- 00:40:27 规模效应和数据
- 00:41:26 ML Infra vs Data Infra
- 00:44:47 对新入行Data Engineering的建议
- 00:47:28 联系方式
Links:
- Airflow: https://airflow.apache.org/
- Piper: https://eng.uber.com/no-code-workflow-orchestrator/
- uWorc: https://eng.uber.com/no-code-workflow-orchestrator/
- Amundsen: https://eng.lyft.com/open-sourcing-amundsen-a-data-discovery-and-metadata-platform-2282bb436234
- PII: https://en.wikipedia.org/wiki/Personal_data
- GDPR: https://en.wikipedia.org/wiki/General_Data_Protection_Regulation
- ETL: https://en.wikipedia.org/wiki/Extract,_transform,_load
- 联系泉来：mail@quanlai.li
联系方式：
- 官网: eng.cafe/
- 微信公众号: Eng Cafe
- Twitter: @engcafefm
- Youtube: Eng Cafe
- Email: hi@eng.cafe
收听渠道:
16 episodes
Share
