Who is using Apache Celeborn?

[Nicholas Jiang]

Search before asking

• I searched in the issues and found nothing similar.

Motivation

Who's using Apache Celeborn?

The Apache Celeborn community grows with the combined efforts of every user and developer and appreciates every user and developer who uses and cares about Celeborn. We are committed to listening to each one of you to make the community more prosperous.

Therefore, the purpose of this topic is to encourage you, as a user or developer, to leave your real-world scenarios so that we can subsequently explore new and better solutions, schedule the releases, and solve real-world problems based on these scenarios.

We look forward to your comments in the comments space, including but not limited to the following areas.

• Organization: e.g. Alibaba Corp.
• Region: e.g. Hangzhou, China
• Contact: e.g. [e-mail]
• Usage Scenario: e.g., we use Apache Celeborn to improve the efficiency and elasticity of different map-reduce engines and provides an elastic, high-efficient management service for intermediate data including shuffle data, spilled data, result data, etc.

Regards,

Apache Celeborn Community

Solution

No response

Anything else?

No response

Are you willing to submit a PR?

• I'm willing to submit a PR!

[gaochao0509]

Organization: New Baatur
Region: Shen zhen, China
Contact information: [e-mail]
Usage scenario: Use Apache Celeborn to improve the efficiency and resilience of the Spark/MapReduce engine, providing elastic and efficient management services for intermediate data such as Shuffle data, overflow data, and result data.
Location: Currently, 10+locations have been deployed, with significant stability improvement and performance improvement of over 30%.

[Yu Li]

Region: Xi'an, China Contact information: [e-mail] Usage scenario: Use Apache Celeborn to improve the efficiency and resilience of the Spark/MapReduce engine, providing elastic and efficient management services for intermediate data such as Shuffle data, overflow data, and result data. Location: Currently, 10+locations have been deployed, with significant stability improvement and performance improvement of over 30%.

@gaochao0509 Thanks for sharing. Mind also share the company/organization? :-)

[cxzl25]

• Organization: Trip.com Group
• Region: Shanghai, China
• Contact: [e-mail]
• Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle.

[alex]

Organization: inmobi Group
Region: Beijing, China
Contact: [e-mail]
Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle.

[RuiQin]

Organization: zto.com Group
Region: Shanghai, China
Contact: [e-mail]
Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle

[PLASH SPEED]

Organization: shuyun Inc
Region: hangzhou, China
Contact: [e-mail]
Usage Scenario: By using apache celeborn, We realised the separation of storage and compute in HIVE-MR3/SPARK-KYUUBI, which effectively reduces the cost of the server room.

[Nicholas Jiang]

• Organization: Bilibili Inc
• Region: Shanghai, China
• Contact: [e-mail]
• Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark and Flink(Batch) engine shuffle.
  

[jiaoqingbo]

• Organization: AsiaInfo
• Region: Beijing, China
• Contact: [e-mail]
• Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle.
  

[moslifulong]

Organization: Zhihu Inc
Region: Beijing, China
Contact: [e-mail]
Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark engine shuffle.

[TongWei]

Organization: Xiaohongshu Inc
Region: Shanghai, China
Contact: [e-mail]
Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark and Flink(Batch) engine shuffle.

[Leo Li]

• Organization: BIGO
• Region: Guangzhou, China
• Contact: [e-mail]
• Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle.
  

[Angerszhuuuu]

Organization: SHOPEE
Region: Shenzhen, China
Contact: [e-mail]
Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle.

[Lianne Li]

Organization: DMALL
Region: Beijing, China
Contact: [e-mail], [e-mail], [e-mail]
Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle.

[camper42]

Organization: Douban
Region: Beijing, China
Contact: [e-mail]
Usage Scenario: we use Apache Celeborn to support better Spark dynamic resource allocation in Kubernetes.

[wfxxh]

Organization: WanFang Data
Region: Beijing, China
Contact: [e-mail]
Usage Scenario: We use Apache Celeborn to improve the stability and efficiency of Spark shuffle on k8s

[吴祥平]

Organization: Tongcheng Travel（同程旅行）.
Region: Jiangsu, China
Contact: www.ly.com
Usage Scenario: we use Apache Celeborn to improve the stability and efficiency of spark shuffle.

[Nan Zhu]

Organization: e.g. Pinterest
Region: US
Contact: @CodingCat
Usage Scenario: we use Apache Celeborn to improve the efficiency and elasticity of Spark@K8S

[Aravind Patnam]

Organization: LinkedIn
Region: US
Contact: @akpatnam25
Usage Scenario: we use Apache Celeborn to improve the stability and efficiency of spark shuffle.

[liugs0213]

Organization: BOSS Zhipin
Region: Beijing, China
Contact: @liugs0213
Usage Scenario: we use Apache Celeborn to improve the efficiency and elasticity of Spark@K8S