Materials regarding decentralized AI research.
This includes literatures on consensus optimization (mainly from mainly control community). Hopefully one can borrow theory from them to build fully decentralized algorithms.
-
Dual averaging for distributed optimization: Convergence analysis and network scaling
2012 IEEE Transactions on Automatic control. Duchi, John C and Agarwal, Alekh and Wainwright, Martin J -
On the convergence of decentralized gradient descent.
2016 SIAM Journal on Optimization. Yuan, Kun and Ling, Qing and Yin, Wotao -
DSA: Decentralized double stochastic averaging gradient algorithm
2016 JMLR. Mokhtari, Aryan and Ribeiro, Alejandro.
-
Can decentralized algorithms outperform centralized algorithms? a case study for decentralized parallel stochastic gradient descent.
NIPS 2017. Xiangru Lian, Ce Zhang, Huan Zhang, Cho-Jui Hsieh, Wei Zhang, and Ji Liu -
D^2: Decentralized training over decentralized data.
ICML 2018. Hanlin Tang, Xiangru Lian, Ming Yan, Ce Zhang, and Ji Liu.
- COLA: Decentralized Linear Learning
NIPS 2018. *Lie He, *An Bian, Martin Jaggi
This is one recent setting proposed to solve one existing decentralized data setting, but the algorithms therein are still centralized algorithms.
- Communication-Efficient Learning of Deep Networks from Decentralized Data
2017 AISTATS. McMahan, Brendan and Moore, Eider and Ramage, Daniel and Hampson, Seth and y Arcas, Blaise Aguera
This is one emerging setting where one wants to learn an intelligent model for mobile devices, such as mobile phones, tablets etc.
-
Randomized gossip algorithms
2006 EEE transactions on information theory. Boyd, Stephen and Ghosh, Arpita and Prabhakar, Balaji and Shah, Devavrat -
Gossip algorithms
2009 Foundations and Trends in Networking. Shah, Devavrat
-
Bitcoin: A Peer-to-Peer Electronic Cash System.
Satoshi Nakamoto. 2008.
https://bitcoin.org/bitcoin.pdf -
When A Small Leak Sinks A Great Ship: Deanonymizing Tor Hidden Service Users Through Bitcoin Transactions Analysis.
Jawaheri, Husam Al and Sabah, Mashael Al and Boshmaf, Yazan and Erbad, Aimen. 2018.
https://arxiv.org/pdf/1801.07501.pdf?utm_source=securitydailynews.com
- Ethereum: A Secure Decentralized Generalised Transaction Leger
Wood, Gavin. 2014.
http://gavwood.com/paper.pdf
- IPFS-content addressed, versioned, P2P file system.
Benet, Juan. 2014.
https://ipfs.io/ipfs/QmR7GSQM93Cx5eAg6a6yRzNde1FQv7uL6X1o4k7zrJa3LX/ipfs.draft3.pdf