Zheng Chen - Assistant Professor Communication systems
Date: - Location: Eurecom
Abstract: Consensus-based decentralized stochastic gradient descent (D-SGD) is a widely adopted algorithm for decentralized training of machine learning models across networked agents. A crucial aspect of D-SGD and its variants is the consensus-based model averaging among agents, which heavily relies on information exchange and fusion within the network. Specifically, for consensus averaging over wireless networks, communication coordination is necessary to determine when and how a node can access the channel and transmit (or receive) information to (or from) its neighbors. This talk focuses on the error-vs-runtime tradeoff in D-SGD over wireless networks, taking into account the actual communication delay per iteration using broadcast transmission for information exchange among locally connected nodes. Two communication approaches are considered: (1) collision-free and (2) random access. For the collision-free approach, we introduce a broadcast-based subgraph sampling (BASS) method, motivated by the recognition that “not all nodes are equally important in a network”. In every consensus iteration, multiple subsets of non-interfering nodes are randomly activated to broadcast model updates to their neighbors under a given communication cost (e.g., number of transmission slots per iteration). The mixing matrices of the activated subgraphs and the sampling probabilities are further optimized for convergence acceleration. For the random access approach, based on a simple probabilistic medium access protocol featuring a “success or collision” communication model, we demonstrate the effects of access probability design and retransmissions. Biography: Zheng Chen is an Associate Professor with the Department of Electrical Engineering at Linköping University, Sweden. She received her Ph.D. degree in 2016 from CentraleSupélec, Université Paris-Saclay, France. She has conducted research on various topics in wireless communication systems, including wireless edge caching, energy harvesting, age of information, and massive MIMO. Her current research focuses on distributed information processing and machine learning over wireless networks. She received the 2020 IEEE Communications Society Young Author Best Paper Award, has served as the co-chair of several workshops and special sessions at IEEE Globecom, ICASSP, and Asilomar, and is currently an Editor of the IEEE Transactions on Green Communications and Networking.
