Autonomous Robot Navigation: Real-time Scene Segmentation (PDF)

Real-time Scene Segmentation Using a Light Deep Neural Network Architecture for Autonomous Robot Navigation on Construction Sites Khashayar Asadi. Ph.D. Student, SM.ASCE1, Pengyu Chen. MSCS Student2, Kevin Han Ph.D. M.ASCE3, Tianfu Wu4, and Edgar Lobato - Detailed Explanation of the Paper: "Real-t...

Real-time Scene Segmentation Using a Light Deep Neural Network Architecture for Autonomous Robot Navigation on Construction Sites Khashayar Asadi. Ph.D. Student, SM.ASCE1, Pengyu Chen. MSCS Student2, Kevin Han Ph.D. M.ASCE3, Tianfu Wu4, and Edgar Lobato - Detailed Explanation of the Paper: "Real-time Scene Segmentation Using a Light Deep Neural Network Architecture for Autonomous Robot Navigation on Construction Sites" 1. Overview & Purpose of the Study The paper presents a lightweight and ef cient deep neural network designed to perform real- time scene segmentation for autonomous robot navigation on construction sites. Key challenges: Construction sites are dynamic and require precise navigation for Unmanned Vehicles (UVs). Standard semantic segmentation models are too large and computationally heavy for mobile robots with limited processing power. The goal is to reduce the model size while maintaining segmentation accuracy, enabling real-time operation on embedded devices (Jetson TX1). 2. Motivation The construction industry has struggled to improve productivity compared to manufacturing. Automated monitoring using Unmanned Vehicles (UVs) can improve data collection. However, existing robotic platforms require separate computational resources for SLAM (Simultaneous Localization and Mapping) and Scene Segmentation, increasing hardware size and power consumption. The previous model (ENet) was optimized for mobile robotics but had a high computational cost, forcing the robot to move slowly for real-time processing. The new proposed model aims to reduce computation time, improve ef ciency, and maintain accuracy. 3. Proposed Solution The paper introduces a lightweight deep learning model optimized for real-time image segmentation to classify navigable space in an autonomous robot's video stream. Key Contributions: 1. Optimized Convolutional Neural Network (CNN) ◦ Uses Depthwise Separable Convolution to reduce computation while maintaining accuracy. ◦ Reduces computational cost by a factor of ~8 compared to standard convolution layers. 2. Factorized Convolution Block (Inspired by MobileNet) ◦ Factorizes convolution into two steps: ▪ Depthwise Convolution (applies a single lter per input channel). fi fi fi  ▪ Pointwise Convolution (1x1 kernel to combine outputs). ◦ Leads to signi cant reduction in computational cost. 3. Network Compression ◦ Uses L1 norm pruning to remove redundant lters. ◦ Filters with low activation values are removed to reduce model size without major accuracy loss. 4. Custom Dataset for Construction Sites ◦ 1000 images collected from construction sites, parking lots, and roads. ◦ Uses Cityscapes dataset for additional training. ◦ Data was pixel-annotated into two classes (ground vs. not ground). 4. Architecture of the Model The new model follows an encoder-decoder architecture, inspired by ENet, but with major optimizations. 1. Encoder ◦ Uses Factorized Convolution Blocks to extract features. ◦ Initial 3×3 convolution with MaxPooling to downsample early. ◦ Skip connections ensure ef cient gradient ow. 2. Decoder ◦ Uses Upsample and LastConv blocks (similar to ENet). ◦ Predicts navigable vs. non-navigable areas in real-time. 5. Experimental Setup & Results The model was implemented using: ◦ TensorFlow & Keras for training. ◦ CUDA & cuDNN for GPU acceleration. ◦ NVIDIA Jetson TX1 for embedded deployment. Training & Testing Details Training: ◦ Used Cityscapes dataset & custom construction site dataset. ◦ Adam Optimizer for fast convergence. ◦ Cross-entropy loss for classi cation. Testing: ◦ Used 150 images from parking lots, roads, and construction sites. ◦ Compared performance of proposed model vs. ENet. Results Accuracy Inference Time Model Model Size Reduction FPS Improvement Loss Reduction Proposed -2% 50% smaller (2225 KB → +5 FPS (from 22 to 18% faster Model accuracy 1068 KB) 27 FPS) fi fi fi fl fi  Key Observations: Only 2% accuracy loss despite major model compression. Inference time decreased by 18%, allowing the robot to process frames faster. Frame rate improved from 22 FPS (ENet) to 27 FPS, making real-time navigation more ef cient. 6. Conclusion & Key Takeaways Signi cant Reduction in Model Size ◦ The model is 50% smaller, allowing multiple modules (SLAM + segmentation) to run on a single Jetson TX1. ◦ Reduces the need for multiple Jetson boards, decreasing hardware cost and power consumption. Real-time Performance on Embedded Devices ◦ The new model achieves 27 FPS, enabling smooth robot navigation. ◦ Lower computation cost reduces system latency. Scalability for Future Autonomous Robotics ◦ The proposed method can be extended to Unmanned Aerial Vehicles (UAVs) and autonomous ground vehicles (UGVs). ◦ Optimized deep learning models are essential for mobile robots operating in computationally constrained environments. 7. Future Improvements Suggested 1. Improve segmentation accuracy ◦ Using attention mechanisms or transformer-based models to improve detection quality. 2. Optimize model compression ◦ Using quantization-aware training (QAT) to further reduce model size without accuracy loss. 3. Integrate additional datasets ◦ Training with more diverse construction site datasets to improve generalization. 4. Deploy on different hardware ◦ Testing performance on Jetson Xavier NX and Jetson Orin for higher ef ciency. Final Thoughts This paper presents an innovative, lightweight deep learning model for real-time scene segmentation in autonomous construction robots. It successfully reduces model size and inference time while maintaining high accuracy, enabling ef cient deployment on low-power embedded devices. Let me know if you need a deeper explanation of any section! 🚀 fi fi fi fi  Proposed Solution: Making the Model Faster and Lighter The researchers developed a lightweight deep learning model to help autonomous robots recognize navigable paths in real-time using a camera. This is important because construction sites are dynamic and complex, and robots need to navigate ef ciently without relying on heavy computing power. To achieve this, the paper introduces four key improvements: 1. Optimized CNN (Making Computation Faster) What’s the problem? Regular convolutional neural networks (CNNs) are computationally expensive, making them hard to run on small robots with limited processing power. Solution: They used a technique called Depthwise Separable Convolution, which breaks down a normal convolution into two simpler steps: 1. Depthwise Convolution – Applies a lter to each input channel separately instead of all at once. 2. Pointwise Convolution (1x1 convolution) – Combines the outputs from the previous step. Why is this helpful? It reduces computational cost by 8 times while keeping accuracy high. 2. Factorized Convolution Block (Inspired by MobileNet) What’s the problem? Traditional convolution layers use more memory and processing power. Solution: Instead of using standard convolutions, they split them into smaller operations, making them faster and more ef cient. How it works: ◦ First, a 1×1 convolution reduces the number of features. ◦ Then, Depthwise Convolution applies lters separately to each channel. ◦ Finally, another 1×1 convolution expands the features again. Why is this helpful? This structure signi cantly reduces the number of computations while still keeping the important image details. 3. Network Compression (Removing Unnecessary Parts) What’s the problem? Deep learning models often have redundant or unimportant lters that do not contribute much to the nal output. Solution: They applied L1 norm pruning, a technique that: ◦ Measures how important each lter is (by calculating its L1 norm). ◦ Removes lters with low importance to reduce the model size without losing accuracy. Why is this helpful? The model becomes smaller and more ef cient, making it easier to run on an embedded device like Jetson TX1. 4. Custom Dataset for Construction Sites (Better Training Data) fi fi fi fi fi fi fi fi fi fi  What’s the problem? Existing datasets (like Cityscapes) don’t have enough construction- speci c images for training. Solution: They created a new dataset with 1,000 images taken from: ◦ Construction sites ◦ Parking lots ◦ Roads How the data was prepared: ◦ They manually labeled each pixel into two categories: 1. Ground (navigable space) 2. Not ground (obstacles, buildings, equipment, etc.) Why is this helpful? The robot learns better by training on real construction environments. Final Takeaway These four improvements make the model: ✅ Faster (by reducing computations) ✅ Lighter (by removing unnecessary parts) ✅ More accurate (by training on better data) ✅ More practical for real-time robot navigation 🚀 Would you like me to explain any part in even simpler terms? 😊 fi