StitchNet: Distributed On-Device Model Partitioning Over Edge Devices Under Volatile Wireless Links
- 주제(키워드) Neurons , Computational modeling , Parallel processing , Computer architecture , Edge computing , Performance evaluation , Data models , Learning systems , Distributed information systems , On-device AI , distributed learning , model parallelism , edge computing
- 주제(기타) Computer Science, Information Systems; Engineering, Electrical & Electronic; Telecommunications
- 설명문(일반) [Lee, Jiho] Univ Virginia, Dept Comp Sci, Charlottesville, VA 22904 USA; [Cho, Jeihee; Lee, Hyungjune] Ewha Womans Univ, Dept Comp Sci & Engn, Seoul 03760, South Korea
- 관리정보기술 faculty
- 등재 SCIE, SCOPUS
- OA유형 gold
- 발행기관 IEEE-INST ELECTRICAL ELECTRONICS ENGINEERS INC
- 발행년도 2022
- 총서유형 Journal
- URI http://www.dcollection.net/handler/ewha/000000203065
- 본문언어 영어
- Published As https://doi.org/10.1109/ACCESS.2022.3215083
초록/요약
Distributed deep learning architecture can achieve scalable learning and inference capability at resource-constrained edge devices. Although the parallelization-based approaches have actively been investigated in the edge computing context, they are not designed for the devices that are usually wireless and mobile, causing the substantial link and device failure issue. We propose a semi-distributed deep learning architecture, StitchNet, based on model parallelism for volatile wireless edge networks. Our algorithm first classifies a set of effective neurons with a substantial impact on their connected neurons across layers. Then, an opportunistic neuron assignment is employed to ensure the full forward and backward propagation paths by stitching the subsets of the model across the devices with path redundancy via neuron cloning for securing high resilience to network and device uncertainty. Simulation-based experiments demonstrate that StitchNet has achieved high inference quality on visual classification tasks even under the volatile lossy network environment, by making edge devices collaboratively find and stitch their in-ward and out-ward edge paths with a reasonable communication overhead.
more