Update README.md

README.md

@@ -1,45 +1,44 @@
-```yaml
 ---
-title: Duino-Idar: Interactive Indoor 3D Mapping via Mobile Video with Semantic Enrichment
-emoji: ✨
-colorFrom: indigo
-colorTo: blue
-sdk: gradio
-sdk_version: 4.x
-app_file: app.py  # Replace with your actual Gradio app file if you have one
+model-index:
+- name: Duino-Idar
+  paper: https://huggingface.co/Duino/Duino-Idar/blob/main/README.md # Link to this README.md
+  results:
+  - task:
+      type: 3D Indoor Mapping # Task type as object
+    dataset:
+      name: Mobile Video # Dataset name as object
+      type: Video # Added dataset type here to resolve the error
+    metrics:
+      - name: Qualitative 3D Reconstruction
+        type: Visual Inspection
+        value: "Visually Inspected; Subjectively assessed for geometric accuracy and completeness of the point cloud." # Added value
+      - name: Semantic Accuracy (Conceptual)
+        type: Qualitative Assessment
+        value: "Qualitatively Assessed; Subjectively evaluated for the relevance and coherence of semantic labels generated for indoor scenes." # Added value
+language: en
+license: mit
 tags:
 - 3d-mapping
-- indoor-reconstruction
 - depth-estimation
 - semantic-segmentation
 - vision-language-model
+- indoor-scene-understanding
 - mobile-video
-- gradio
-- point-cloud
 - dpt
 - paligemma
-- computer-vision
-- research-paper
+- gradio
+- point-cloud
 author: Jalal Mansour (Jalal Duino)
-date: 2025-02-18
+date_created: 2025-02-18
 email: [email protected]
-hf_space: Duino # Assuming 'Duino' is your HF username/space name
-license: mit
+hf_hub_url: https://huggingface.co/Duino/Duino-Idar
 ---
 
-# Duino-Idar: Interactive Indoor 3D Mapping via Mobile Video with Semantic Enrichment
+# Duino-Idar: An Interactive Indoor 3D Mapping System via Mobile Video with Semantic Enrichment
 
-**Author:** Jalal Mansour (Jalal Duino)
-**Date Created:** 2025-02-18
-**Email:** [[email protected]](mailto:[email protected])
-**Hugging Face Space:** [https://huggingface.co/Duino](https://huggingface.co/Duino)
-**License:** MIT License
+**Abstract**
 
----
-
-## Abstract
-
-This paper introduces Duino-Idar, a novel end-to-end system for generating interactive 3D maps of indoor environments from mobile video.  Leveraging state-of-the-art monocular depth estimation techniques, specifically DPT (Dense Prediction Transformer)-based models, and semantic understanding via a fine-tuned vision-language model (PaLiGemma), Duino-Idar offers a comprehensive solution for indoor scene reconstruction. The system extracts key frames from video input, computes depth maps, constructs a 3D point cloud, and enriches it with semantic labels. A user-friendly Gradio-based graphical user interface (GUI) facilitates video upload, processing, and interactive 3D scene exploration.  This research details the system's architecture, implementation, and potential applications in areas such as indoor navigation, augmented reality, and automated scene understanding, setting the stage for future enhancements including LiDAR integration for improved accuracy and robustness.
+This paper introduces Duino-Idar, a novel end-to-end system for generating interactive 3D maps of indoor environments from mobile video. Leveraging state-of-the-art monocular depth estimation techniques, specifically DPT (Dense Prediction Transformer)-based models, and semantic understanding via a fine-tuned vision-language model (PaLiGemma), Duino-Idar offers a comprehensive solution for indoor scene reconstruction. The system extracts key frames from video input, computes depth maps, constructs a 3D point cloud, and enriches it with semantic labels. A user-friendly Gradio-based graphical user interface (GUI) facilitates video upload, processing, and interactive 3D scene exploration. This research details the system's architecture, implementation, and potential applications in areas such as indoor navigation, augmented reality, and automated scene understanding, setting the stage for future enhancements including LiDAR integration for improved accuracy and robustness.
 
 **Keywords:** 3D Mapping, Indoor Reconstruction, Mobile Video, Depth Estimation, Semantic Segmentation, Vision-Language Models, DPT, PaLiGemma, Point Cloud, Gradio, Interactive Visualization.
 
@@ -49,7 +48,7 @@ This paper introduces Duino-Idar, a novel end-to-end system for generating inter
 
 Recent advancements in computer vision and deep learning have significantly propelled the field of 3D scene reconstruction from 2D imagery. Mobile devices, now ubiquitous and equipped with high-quality cameras, provide a readily available source of video data suitable for spatial mapping. While monocular depth estimation has matured considerably, enabling real-time applications, many existing 3D reconstruction approaches lack a crucial component: semantic understanding of the scene. This semantic context is vital for enabling truly interactive and context-aware applications, such as augmented reality (AR) navigation, object recognition, and scene understanding for robotic systems.
 
-To address this gap, we present Duino-Idar, an innovative system that integrates a robust depth estimation pipeline with a fine-tuned vision-language model, PaLiGemma, to enhance indoor 3D mapping.  The system's name, Duino-Idar, reflects the vision of combining accessible technology ("Duino," referencing approachability and user-centric design) with advanced spatial sensing ("Idar," hinting at the potential for LiDAR integration in future iterations, although the current prototype focuses on vision-based depth).  This synergistic combination not only achieves geometric reconstruction but also provides semantic enrichment, significantly enhancing both visualization and user interaction capabilities.  This paper details the architecture, implementation, and potential of Duino-Idar, highlighting its contribution to accessible and semantically rich indoor 3D mapping.
+To address this gap, we present Duino-Idar, an innovative system that integrates a robust depth estimation pipeline with a fine-tuned vision-language model, PaLiGemma, to enhance indoor 3D mapping. The system's name, Duino-Idar, reflects the vision of combining accessible technology ("Duino," referencing approachability and user-centric design) with advanced spatial sensing ("Idar," hinting at the potential for LiDAR integration in future iterations, although the current prototype focuses on vision-based depth). This synergistic combination not only achieves geometric reconstruction but also provides semantic enrichment, significantly enhancing both visualization and user interaction capabilities. This paper details the architecture, implementation, and potential of Duino-Idar, highlighting its contribution to accessible and semantically rich indoor 3D mapping.
 
 ---
 
@@ -58,15 +57,19 @@ To address this gap, we present Duino-Idar, an innovative system that integrates
 Our work builds upon and integrates several key areas of research:
 
 ### 2.1 Monocular Depth Estimation:
-The foundation of our geometric reconstruction lies in monocular depth estimation. Models such as MiDaS [1] and DPT [2] have demonstrated remarkable capabilities in inferring depth from single images. DPT, in particular, leverages transformer architectures to capture global contextual information, leading to improved depth accuracy compared to earlier convolutional neural network (CNN)-based methods.  Equation (1) illustrates the depth normalization process used in DPT-like models to scale the predicted depth map to a usable range.
+
+The foundation of our geometric reconstruction lies in monocular depth estimation. Models such as MiDaS [1] and DPT [2] have demonstrated remarkable capabilities in inferring depth from single images. DPT, in particular, leverages transformer architectures to capture global contextual information, leading to improved depth accuracy compared to earlier convolutional neural network (CNN)-based methods. Equation (1) illustrates the depth normalization process used in DPT-like models to scale the predicted depth map to a usable range.
 
 ### 2.2 3D Reconstruction Techniques:
-Generating 3D point clouds or meshes from 2D inputs is a well-established field, encompassing techniques from photogrammetry [3] and Simultaneous Localization and Mapping (SLAM) [4].  Our approach utilizes depth maps derived from DPT to construct a point cloud, offering a simpler yet effective method for 3D scene representation, particularly suitable for indoor environments where texture and feature richness can support monocular depth estimation. The transformation from 2D pixel coordinates to 3D space is mathematically described by the pinhole camera model, as shown in Equations (2)-(4).
+
+Generating 3D point clouds or meshes from 2D inputs is a well-established field, encompassing techniques from photogrammetry [3] and Simultaneous Localization and Mapping (SLAM) [4]. Our approach utilizes depth maps derived from DPT to construct a point cloud, offering a simpler yet effective method for 3D scene representation, particularly suitable for indoor environments where texture and feature richness can support monocular depth estimation. The transformation from 2D pixel coordinates to 3D space is mathematically described by the pinhole camera model, as shown in Equations (2)-(4).
 
 ### 2.3 Vision-Language Models for Semantic Understanding:
+
 Vision-language models (VLMs) have emerged as powerful tools for bridging the gap between visual and textual understanding. PaLiGemma [5] is a state-of-the-art multimodal model that integrates image understanding with natural language processing. Fine-tuning such models on domain-specific datasets, such as indoor scenes, allows for the generation of semantic annotations and descriptions that can be overlaid on reconstructed 3D models, enriching them with contextual information. The fine-tuning process for PaLiGemma, aimed at minimizing the token prediction loss, is formalized in Equation (5).
 
 ### 2.4 Interactive 3D Visualization:
+
 Effective visualization is crucial for user interaction with 3D data. Libraries like Open3D [6] and Plotly [7] provide tools for interactive exploration of 3D point clouds and meshes. Open3D, in particular, offers robust functionalities for point cloud manipulation, rendering, and visualization, making it an ideal choice for desktop-based interactive 3D scene exploration. For web-based interaction, Plotly offers excellent capabilities for embedding interactive 3D visualizations within web applications.
 
 ---
@@ -77,9 +80,11 @@ Effective visualization is crucial for user interaction with 3D data. Libraries
 
 The Duino-Idar system is structured into three primary modules, as illustrated in Figure 1:
 
-1.  **Video Processing and Frame Extraction:**  This module ingests mobile video input and extracts representative key frames at configurable intervals to reduce computational redundancy and capture scene changes effectively.
-2.  **Depth Estimation and 3D Reconstruction:**  Each extracted frame is processed by a DPT-based depth estimator to generate a depth map. These depth maps are then converted into 3D point clouds using a pinhole camera model, transforming 2D pixel coordinates into 3D spatial positions.
-3.  **Semantic Enrichment and Visualization:** A fine-tuned PaLiGemma model provides semantic annotations for the extracted key frames, enriching the 3D reconstruction with object labels and scene descriptions.  A Gradio-based GUI integrates these modules, providing a user-friendly interface for video upload, processing, interactive 3D visualization, and exploration of the semantically enhanced 3D scene.
+1.  **Video Processing and Frame Extraction:** This module ingests mobile video input and extracts representative key frames at configurable intervals to reduce computational redundancy and capture scene changes effectively.
+2.  **Depth Estimation and 3D Reconstruction:** Each extracted frame is processed by a DPT-based depth estimator to generate a depth map. These depth maps are then converted into 3D point clouds using a pinhole camera model, transforming 2D pixel coordinates into 3D spatial positions.
+3.  **Semantic Enrichment and Visualization:** A fine-tuned PaLiGemma model provides semantic annotations for the extracted key frames, enriching the 3D reconstruction with object labels and scene descriptions. A Gradio-based GUI integrates these modules, providing a user-friendly interface for video upload, processing, interactive 3D visualization, and exploration of the semantically enhanced 3D scene.
+
+**Figure 1: System Architecture Diagram**
 
 ```mermaid
 graph LR
@@ -99,7 +104,7 @@ graph LR
     style H fill:#eee,stroke:#333,stroke-width:2px
     style I fill:#ace,stroke:#333,stroke-width:2px
 ```
-*Figure 1: Duino-Idar System Architecture.  The diagram illustrates the flow of data through the system modules, from video input to interactive 3D visualization with semantic enrichment.*
+*Figure 1: Duino-Idar System Architecture. The diagram illustrates the flow of data through the system modules, from video input to interactive 3D visualization with semantic enrichment.*
 
 ### 3.2 Detailed Pipeline
 
@@ -111,20 +116,20 @@ The Duino-Idar pipeline operates through the following detailed steps:
 
 2.  **Depth Estimation Module:**
     *   **Preprocessing:** Each extracted frame undergoes preprocessing, including resizing and normalization, to optimize it for input to the DPT model. This ensures consistent input dimensions and value ranges for the depth estimation network.
-    *   **Depth Prediction:**  The preprocessed frame is fed into the DPT model, which generates a depth map. This depth map represents the estimated distance of each pixel in the image from the camera.
-    *   **Normalization and Scaling:** The raw depth map is normalized to a standard range (e.g., 0-1 or 0-255) for subsequent 3D reconstruction and visualization.  Equation (1) details the normalization process.
+    *   **Depth Prediction:** The preprocessed frame is fed into the DPT model, which generates a depth map. This depth map represents the estimated distance of each pixel in the image from the camera.
+    *   **Normalization and Scaling:** The raw depth map is normalized to a standard range (e.g., 0-1 or 0-255) for subsequent 3D reconstruction and visualization. Equation (1) details the normalization process.
 
 3.  **3D Reconstruction Module:**
-    *   **Point Cloud Generation:**  A pinhole camera model is applied to convert the depth map and corresponding pixel coordinates into 3D coordinates in camera space. Color information from the original frame is associated with each 3D point to create a colored point cloud. Equations (2), (3), and (4) formalize this transformation.
-    *   **Point Cloud Aggregation:** To build a comprehensive 3D model, point clouds generated from multiple key frames are aggregated. In this initial implementation, we assume a static camera or negligible inter-frame motion for simplicity.  More advanced implementations could incorporate camera pose estimation and point cloud registration for improved accuracy, especially in dynamic scenes. The aggregation process is mathematically represented by Equation (4).
+    *   **Point Cloud Generation:** A pinhole camera model is applied to convert the depth map and corresponding pixel coordinates into 3D coordinates in camera space. Color information from the original frame is associated with each 3D point to create a colored point cloud. Equations (2), (3), and (4) formalize this transformation.
+    *   **Point Cloud Aggregation:** To build a comprehensive 3D model, point clouds generated from multiple key frames are aggregated. In this initial implementation, we assume a static camera or negligible inter-frame motion for simplicity. More advanced implementations could incorporate camera pose estimation and point cloud registration for improved accuracy, especially in dynamic scenes. The aggregation process is mathematically represented by Equation (4).
 
 4.  **Semantic Enhancement Module:**
-    *   **Vision-Language Processing:**  The fine-tuned PaLiGemma model processes the key frames to generate scene descriptions and semantic labels. The model is prompted to identify objects and provide contextual information relevant to indoor scenes.
+    *   **Vision-Language Processing:** The fine-tuned PaLiGemma model processes the key frames to generate scene descriptions and semantic labels. The model is prompted to identify objects and provide contextual information relevant to indoor scenes.
     *   **Semantic Data Integration:** Semantic labels generated by PaLiGemma are overlaid onto the reconstructed point cloud. This integration can be achieved through various methods, such as associating semantic labels with clusters of points or generating bounding boxes around semantically labeled objects within the 3D scene.
 
 5.  **Visualization and User Interface Module:**
     *   **Interactive 3D Viewer:** The final semantically enriched 3D model is visualized using Open3D (or Plotly for web-based deployments). Users can interact with the 3D scene, rotating, zooming, and panning to explore the reconstructed environment.
-    *   **Gradio GUI:**  A user-friendly Gradio web interface provides a seamless experience, allowing users to upload videos, initiate the processing pipeline, and interactively navigate the resulting 3D scene. The GUI also provides controls for adjusting parameters like frame extraction interval and potentially visualizing semantic labels.
+    *   **Gradio GUI:** A user-friendly Gradio web interface provides a seamless experience, allowing users to upload videos, initiate the processing pipeline, and interactively navigate the resulting 3D scene. The GUI also provides controls for adjusting parameters like frame extraction interval and potentially visualizing semantic labels.
 
 ---
 
@@ -136,43 +141,43 @@ The Duino-Idar system relies on several core mathematical principles:
 
 **1. Depth Estimation via Deep Network:**
 
-Let $I \in \mathbb{R}^{H \times W \times 3}$ represent the input image of height $H$ and width $W$.  The DPT model, denoted as $f$, with learnable parameters $\theta$, estimates the depth map $D$:
+Let $I \in \mathbb{R}^{H \times W \times 3}$ represent the input image of height $H$ and width $W$. The DPT model, denoted as $f$, with learnable parameters $\theta$, estimates the depth map $D$:
 
-**(1)  $D = f(I; \theta)$**
+**(1)**  $D = f(I; \theta)$
 
 The depth map $D$ is then normalized to obtain $D_{\text{norm}}$:
 
-**(2)  $D_{\text{norm}}(u,v) = \frac{D(u,v)}{\displaystyle \max_{(u,v)} D(u,v)}$**
+**(2)**  $D_{\text{norm}}(u,v) = \frac{D(u,v)}{\displaystyle \max_{(u,v)} D(u,v)}$
 
 If a maximum physical depth $Z_{\max}$ is assumed, the scaled depth $z(u,v)$ is:
 
-**(3)  $z(u,v) = D_{\text{norm}}(u,v) \times Z_{\max}$**
+**(3)**  $z(u,v) = D_{\text{norm}}(u,v) \times Z_{\max}$
 
 For practical implementation and visualization, we often scale the depth to an 8-bit range:
 
-**(4)  $D_{\text{scaled}}(u,v) = \frac{D(u,v)}{\displaystyle \max_{(u,v)} D(u,v)} \times 255$**
+**(4)**  $D_{\text{scaled}}(u,v) = \frac{D(u,v)}{\displaystyle \max_{(u,v)} D(u,v)} \times 255$
 
 **2. 3D Reconstruction with Pinhole Camera Model:**
 
 Assuming a pinhole camera model with intrinsic parameters: focal lengths $(f_x, f_y)$ and principal point $(c_x, c_y)$, the intrinsic matrix $K$ is:
 
-**(5)  $K = \begin{pmatrix}
+**(5)**  $K = \begin{pmatrix}
 f_x & 0 & c_x \\
 0 & f_y & c_y \\
 0 & 0 & 1
-\end{pmatrix}$**
+\end{pmatrix}$
 
 Given a pixel $(u, v)$ and its depth value $z(u,v)$, the 3D coordinates $(x, y, z)$ in the camera coordinate system are:
 
-**(6)  $x = \frac{(u - c_x) \cdot z(u,v)}{f_x}$**
+**(6)**  $x = \frac{(u - c_x) \cdot z(u,v)}{f_x}$
 
-**(7)  $y = \frac{(v - c_y) \cdot z(u,v)}{f_y}$**
+**(7)**  $y = \frac{(v - c_y) \cdot z(u,v)}{f_y}$
 
-**(8)  $z = z(u,v)$**
+**(8)**  $z = z(u,v)$
 
 In matrix form:
 
-**(9)  $\begin{pmatrix}
+**(9)**  $\begin{pmatrix}
 x \\
 y \\
 z
@@ -181,13 +186,13 @@ z
 u \\
 v \\
 1
-\end{pmatrix}$**
+\end{pmatrix}$
 
 **3. Aggregation of Multiple Frames:**
 
 Let $P_i$ be the point cloud from the $i^{th}$ frame, where $P_i = \{(x_{i,j}, y_{i,j}, z_{i,j}) \mid j = 1, 2, \ldots, N_i\}$. The overall point cloud $P$ is the union:
 
-**(10) $P = \bigcup_{i=1}^{M} P_i$**
+**(10)** $P = \bigcup_{i=1}^{M} P_i$
 
 where $M$ is the number of frames.
 
@@ -195,7 +200,7 @@ where $M$ is the number of frames.
 
 For fine-tuning PaLiGemma, given an image $I$ and caption tokens $c = (c_1, c_2, \ldots, c_T)$, the cross-entropy loss $\mathcal{L}$ is minimized:
 
-**(11) $\mathcal{L} = -\sum_{t=1}^{T} \log P(c_t \mid c_{<t}, I)$**
+**(11)** $\mathcal{L} = -\sum_{t=1}^{T} \log P(c_t \mid c_{<t}, I)$
 
 where $P(c_t \mid c_{<t}, I)$ is the conditional probability of predicting the $t^{th}$ token given the preceding tokens $c_{<t}$ and the input image $I$.
 
@@ -217,7 +222,7 @@ pip install transformers peft bitsandbytes gradio opencv-python pillow numpy tor
 
 ### 4.3 Code Snippets and Dynamicity
 
-Here are illustrative code snippets demonstrating key functionalities.  These are excerpts from the provided datasheet code and are used for demonstration purposes within this paper.
+Here are illustrative code snippets demonstrating key functionalities. These are excerpts from the provided datasheet code and are used for demonstration purposes within this paper.
 
 #### 4.3.1 Depth Estimation using DPT:
 
@@ -289,24 +294,58 @@ def visualize_3d_model(ply_file):
     pcd = o3d.io.read_point_cloud(ply_file)
     o3d.visualization.draw_geometries([pcd]) # Interactive window
 
+def process_video(video_path):
+    """ Process video: extract frames, estimate depth, and generate a 3D model. """
+    frames = extract_frames(video_path) # Assuming extract_frames function is defined (from full code)
+    depth_maps = [estimate_depth(frame) for frame in frames]
+    final_pcd = None
+    for frame, depth_map in zip(frames, depth_maps):
+        pcd = reconstruct_3d(depth_map, frame)
+        if final_pcd is None:
+            final_pcd = pcd
+        else:
+            final_pcd += pcd
+    o3d.io.write_point_cloud("output.ply", final_pcd)
+    return "output.ply"
+
+def extract_frames(video_path, interval=10): # Example extract_frames function
+    import cv2
+    from PIL import Image
+    cap = cv2.VideoCapture(video_path)
+    frames = []
+    i = 0
+    while cap.isOpened():
+        ret, frame = cap.read()
+        if not ret:
+            break
+        if i % interval == 0:
+            frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+            frames.append(Image.fromarray(frame))
+        i += 1
+    cap.release()
+    return frames
+
+
 with gr.Blocks() as demo:
     gr.Markdown("### Duino-Idar 3D Mapping")
     video_input = gr.Video(label="Upload Video", type="filepath")
     process_btn = gr.Button("Process & Visualize")
-    # ... (integrate process_video function from datasheet here) ...
+    output_file = gr.File(label="Generated 3D Model (PLY)")
+
+    process_btn.click(fn=process_video, inputs=video_input, outputs=output_file)
+
+    view_btn = gr.Button("View 3D Model")
+    view_btn.click(fn=visualize_3d_model, inputs=output_file, outputs=None)
 
-    process_btn.click(fn=process_video, inputs=video_input, outputs=None) # Example interaction
 
 demo.launch()
 ```
 
-This Gradio code demonstrates the dynamic user interface for Duino-Idar.  The `gr.Blocks()` and associated components create an interactive web application. Users can upload videos, trigger the processing pipeline via the "Process & Visualize" button, and the `visualize_3d_model` function (and the integrated `process_video` function which is not fully shown here for brevity, but present in the datasheet) dynamically handles the video processing and 3D visualization when the button is clicked.
-
-*(Figure 2: Conceptual Gradio Interface Screenshot - Conceptual)*
+**Figure 2: Example Gradio Interface Screenshot - Conceptual**
 
-*(A conceptual screenshot of a Gradio interface showing a video upload area, a process button, and potentially a placeholder for 3D visualization or a link to view the 3D model.  Due to limitations of text-based output, a real screenshot cannot be embedded here, but imagine a simple, functional web interface based on Gradio.)*
+*(A conceptual screenshot of a Gradio interface showing a video upload area, a process button, and potentially a placeholder for 3D visualization or a link to view the 3D model. Due to limitations of text-based output, a real screenshot cannot be embedded here, but imagine a simple, functional web interface based on Gradio.)*
 
-*Figure 2: Conceptual Gradio Interface for Duino-Idar.  This illustrates a user-friendly web interface for video input, processing initiation, and 3D model visualization access.*
+*Figure 2: Conceptual Gradio Interface for Duino-Idar. This illustrates a user-friendly web interface for video input, processing initiation, and 3D model visualization access.*
 
 ---
 
@@ -314,30 +353,54 @@ This Gradio code demonstrates the dynamic user interface for Duino-Idar.  The `g
 
 While this paper focuses on system design and implementation, a preliminary demonstration was conducted to validate the Duino-Idar pipeline. Mobile videos of indoor environments (e.g., living rooms, kitchens, offices) were captured using a standard smartphone camera. These videos were then uploaded to the Duino-Idar Gradio interface.
 
+To illustrate the depth estimation performance conceptually, consider a simplified representation of depth accuracy across different distances.  **Note:** Markdown cannot render dynamic graphs. The following is a *text-based* approximation. For actual graphs, you would embed image links below.
+
+**Conceptual Depth Accuracy vs. Distance (Text-Based Graph):**
+
+```
+Depth Accuracy (Qualitative)
+^
+| Excellent
+|     *
+|    * *
+|   *   *
+| Good      *
+|  *         *
+| Moderate        *
++---------------------> Distance from Camera (meters)
+```
+
+*This is a highly simplified, qualitative representation.  For quantitative evaluation, you would typically use metrics like Root Mean Squared Error (RMSE) or Mean Absolute Error (MAE) on a dataset with ground truth depth.*
+
 The system successfully processed these videos, extracting key frames, estimating depth maps using the DPT model, and reconstructing 3D point clouds. The fine-tuned PaLiGemma model provided semantic labels, such as "sofa," "table," "chair," and "window," which were (in a conceptual demonstration, as full integration is ongoing) intended to be overlaid onto the 3D point cloud, enabling interactive semantic exploration.
 
-*(Figure 3: Example 3D Point Cloud Visualization - Conceptual)*
+**Figure 3: Example 3D Point Cloud Visualization - Conceptual**
 
-*(A conceptual visualization of a 3D point cloud generated by Duino-Idar, potentially showing a simple room scene with furniture.  Due to limitations of text-based output, a real 3D rendering cannot be embedded here, but imagine a sparse but recognizable 3D point cloud of a room.)*
+*(A conceptual visualization of a 3D point cloud generated by Duino-Idar, potentially showing a simple room scene with furniture. Due to limitations of text-based output, a real 3D rendering cannot be embedded here, but imagine a sparse but recognizable 3D point cloud of a room.)*
 
 *Figure 3: Conceptual 3D Point Cloud Visualization. This illustrates a representative point cloud output from Duino-Idar, showing the geometric reconstruction of an indoor scene.*
 
+**Figure 4: Semantic Labeling Performance (Conceptual - Image Link)**
+
+[![Semantic Labeling Performance](link-to-your-semantic-labeling-performance-graph.png)](link-to-your-semantic-labeling-performance-graph.png)
+*Figure 4: Conceptual Semantic Labeling Performance.  [Replace `link-to-your-semantic-labeling-performance-graph.png`](link-to-your-semantic-labeling-performance-graph.png) with the actual URL or relative path to an image of a graph illustrating semantic labeling quality, if available. This could be a bar chart showing accuracy per object category, for instance.*
+
 ## 6. Discussion and Future Work
 
 Duino-Idar demonstrates a promising approach to accessible and semantically rich indoor 3D mapping using mobile video. The integration of DPT-based depth estimation and PaLiGemma for semantic enrichment provides a valuable combination, offering both geometric and contextual understanding of indoor scenes. The Gradio interface significantly enhances usability, making the system accessible to users with varying technical backgrounds.
 
 However, several areas warrant further investigation and development:
 
-*   **Enhanced Semantic Integration:**  Future work will focus on robustly overlaying semantic labels directly onto the point cloud, potentially using point cloud segmentation techniques to associate labels with specific object regions. This will enable object-level annotation and more granular scene understanding.
-*   **Multi-Frame Fusion and SLAM:**  The current point cloud aggregation is simplistic. Integrating a robust SLAM or multi-view stereo method is crucial for handling camera motion and improving reconstruction fidelity, particularly in larger or more complex indoor environments.  This would also address potential drift and inconsistencies arising from independent frame processing.
-*   **LiDAR Integration (Duino-*Idar* Vision):**  To truly realize the "Idar" aspect of Duino-Idar, future iterations will explore the integration of LiDAR sensors.  LiDAR data can provide highly accurate depth measurements, complementing and potentially enhancing the video-based depth estimation, especially in challenging lighting conditions or for textureless surfaces.  A hybrid approach combining LiDAR and vision could significantly improve the robustness and accuracy of the system.
-*   **Real-Time Processing and Optimization:**  The current implementation is primarily offline. Optimizations, such as using TensorRT or mobile GPU acceleration, are necessary to achieve real-time or near-real-time mapping capabilities, making Duino-Idar suitable for applications like real-time AR navigation.
-*   **Improved User Interaction:**  Further enhancements to the Gradio interface, or integration with web-based 3D viewers like Three.js, can create a more immersive and intuitive user experience, potentially enabling virtual walkthroughs and interactive object manipulation within the reconstructed 3D scene.
-*   **Handling Dynamic Objects:** The current system assumes static scenes.  Future research should address the challenge of dynamic objects (e.g., people, moving furniture) within indoor environments, potentially using techniques for object tracking and removal or separate reconstruction of static and dynamic elements.
+*   **Enhanced Semantic Integration:** Future work will focus on robustly overlaying semantic labels directly onto the point cloud, potentially using point cloud segmentation techniques to associate labels with specific object regions. This will enable object-level annotation and more granular scene understanding.
+*   **Multi-Frame Fusion and SLAM:** The current point cloud aggregation is simplistic. Integrating a robust SLAM or multi-view stereo method is crucial for handling camera motion and improving reconstruction fidelity, particularly in larger or more complex indoor environments. This would also address potential drift and inconsistencies arising from independent frame processing.
+*   **LiDAR Integration (Duino-*Idar* Vision):** To truly realize the "Idar" aspect of Duino-Idar, future iterations will explore the integration of LiDAR sensors. LiDAR data can provide highly accurate depth measurements, complementing and potentially enhancing the video-based depth estimation, especially in challenging lighting conditions or for textureless surfaces. A hybrid approach combining LiDAR and vision could significantly improve the robustness and accuracy of the system.
+*   **Real-Time Processing and Optimization:** The current implementation is primarily offline. Optimizations, such as using TensorRT or mobile GPU acceleration, are necessary to achieve real-time or near-real-time mapping capabilities, making Duino-Idar suitable for applications like real-time AR navigation.
+*   **Improved User Interaction:** Further enhancements to the Gradio interface, or integration with web-based 3D viewers like Three.js, can create a more immersive and intuitive user experience, potentially enabling virtual walkthroughs and interactive object manipulation within the reconstructed 3D scene.
+*   **Handling Dynamic Objects:** The current system assumes static scenes. Future research should address the challenge of dynamic objects (e.g., people, moving furniture) within indoor environments, potentially using techniques for object tracking and removal or separate reconstruction of static and dynamic elements.
 
 ## 7. Conclusion
 
-Duino-Idar presents a novel and accessible system for indoor 3D mapping from mobile video, enriched with semantic understanding through the integration of deep learning-based depth estimation and vision-language models. By leveraging state-of-the-art DPT models and fine-tuning PaLiGemma for indoor scene semantics, the system achieves both geometric reconstruction and valuable scene context. The user-friendly Gradio interface lowers the barrier to entry, enabling broader accessibility for users to create and explore 3D representations of indoor spaces.  While this initial prototype lays a strong foundation, future iterations will focus on enhancing semantic integration, improving reconstruction robustness through multi-frame fusion and LiDAR integration, and optimizing for real-time performance, ultimately expanding the applicability and user experience of Duino-Idar in diverse domains such as augmented reality, robotics, and interior design.
+Duino-Idar presents a novel and accessible system for indoor 3D mapping from mobile video, enriched with semantic understanding through the integration of deep learning-based depth estimation and vision-language models. By leveraging state-of-the-art DPT models and fine-tuning PaLiGemma for indoor scene semantics, the system achieves both geometric reconstruction and valuable scene context. The user-friendly Gradio interface lowers the barrier to entry, enabling broader accessibility for users to create and explore 3D representations of indoor spaces. While this initial prototype lays a strong foundation, future iterations will focus on enhancing semantic integration, improving reconstruction robustness through multi-frame fusion and LiDAR integration, and optimizing for real-time performance, ultimately expanding the applicability and user experience of Duino-Idar in diverse domains such as augmented reality, robotics, and interior design.
 
 ---
 
@@ -355,35 +418,4 @@ Duino-Idar presents a novel and accessible system for indoor 3D mapping from mob
 
 [6] Zhou, Q. Y., Park, J., & Koltun, V. (2018). Open3D: A modern library for 3D data processing. *arXiv preprint arXiv:1801.09847*.
 
-[7] Plotly Technologies Inc. (2015). *Plotly Python Library*. https://plotly.com/python/
-
----
-
-## Citation
-
-If you use Duino-Idar in your research, please cite the following paper:
-
-```bibtex
-@misc{duino-idar-2025,
-  author = {Jalal Mansour (Jalal Duino)},
-  title = {Duino-Idar: Interactive Indoor 3D Mapping via Mobile Video with Semantic Enrichment},
-  year = {2025},
-  publisher = {Hugging Face Space},
-  howpublished = {Online},
-  url = {https://huggingface.co/Duino/duino-idar} # Assuming 'duino-idar' is your Space name in HF Duino org
-}
-```
-
-## Contact
-
-For questions or collaborations, please contact:
-
-Jalal Mansour (Jalal Duino) - [[email protected]](mailto:[email protected])
-
----
-
-## License
-
-This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details. (You'll need to create a LICENSE file in your repository with the MIT license text).
-
----
+[7] Plotly Technologies Inc. (2015). *Plotly Python Library*. https://plotly.com/python/