Hello peiqi/lfd

README.md

@@ -39,7 +39,7 @@ We recommend the following hardware to run Stretch AI. Other GPUs and other vers
 
 Artificial intelligence (AI) for robots often has complex dependencies, including the need for trained models. Consequently, installing *stretch-ai* from source can be challenging.
 
-First, you will need to install software on your Stretch robot and another computer with a GPU (*GPU computer*). Use the following link to go to the installation instructions: [Instructions for Installing Stretch AI](https://github.com/hello-robot/stretch_ai/blob/main/docs/start_with_docker_plus_virtenv.md)
+First, you will need to install software on your Stretch robot and another computer with a GPU (*GPU computer*). Use the following link to go to the installation instructions: [Instructions for Installing Stretch AI](./docs/start_with_docker_plus_virtenv.md)
 
 Once you've completed this installation, you can start the server on your Stretch robot.  Prior to running the script, you need to have homed your robot with `stretch_robot_home.py`. Then, run the following command:
 

docker/Dockerfile.ros2

@@ -19,8 +19,8 @@ USER hello-robot
 RUN echo "Installing Stretch AI package"
 RUN pip install --upgrade pip setuptools packaging build meson ninja
 # Do this if you want to check out a specific branch; it's important to make sure versions match:
-# RUN git clone https://github.com/hello-robot/stretch_ai
-RUN git clone https://github.com/hello-robot/stretch_ai --branch hello-peiqi/slam_checking
+# RUN git clone https://github.com/hello-robot/stretch_ai --branch main
+RUN git clone https://github.com/hello-robot/stretch_ai
 WORKDIR /home/hello-robot/stretch_ai
 RUN pip install -e src/
 

docs/data_collection.md

@@ -97,9 +97,6 @@ Launch this command from the directory where URDFs are stored (default is root o
 If you have already ran an app with `--robot-ip` flag, such as the `view_images` test app, you can omit the flag in subsequent runs.
 
 ```bash
-# The -s flag enables png images to be saved in addition to videos, which is faster for model training if training is CPU bound (no video decoding)
-
-TASK_NAME=<name-of-task>
 python3 -m stretch.app.dex_teleop.ros2_leader --robot_ip $ROBOT_IP --task-name $TASK_NAME --teleop-mode <teleop-mode> --clutch
 ```
 

docs/install_details.md

@@ -113,16 +113,6 @@ cd ~/ament_ws
 colcon build --packages-select stretch_ros2_bridge
 ```
 
-#### Experimental: Install ORB-SLAM3 On the Robot (Advanced)
-
-[ORB-SLAM3](https://arxiv.org/pdf/2007.11898) is an open-source VSLAM (visual slam) library. Using it in conjunction with LIDAR-based localization can improve performance in many environments. Installation is documented in a [separate file](docs/orbslam3.md).
-
-*Installation is not required to use Stretch AI.* If you chose to do so, you can then then use the ORB-SLAM3 version of the server launch file:
-
-```
-ros2 launch stretch_ros2_bridge server_orbslam3.launch.py
-```
-
 ### Using LLMs
 
 We use many open-source LLMs from [Huggingface](https://huggingface.co/). TO use them, you will need to make sure `transformers` is installed and up to date. You can install it with:

docs/learning_from_demonstration.md

@@ -32,7 +32,9 @@ Our LfD code is a [fork of LeRobot](https://github.com/hello-robot/lerobot). The
 
 ## Format data and push to huggingface repo
 
-### [Authenticate with huggingface-cli](https://huggingface.co/docs/huggingface_hub/en/guides/cli)
+### [Authenticate with huggingface-cli](https://huggingface.co/docs/huggingface_hub/en/guides/cli) 
+
+You should finish this before doing the remaining steps or you might waste 1 hour in uploading dataset to huggingface.
 
 ### Optional: login to weights and biases (wandb)
 

docs/start_with_docker_plus_virtenv.md

@@ -136,10 +136,19 @@ First, check to see if mamba is installed on your computer:
 mamba
 ```
 
-If you get an "command not found" error, then follow the install instructions for mamba here: https://github.com/conda-forge/miniforge#download
+If you get an "command not found" error, then you have not yet installed mamba.
+
+Follow the install instructions for mamba [here](https://github.com/conda-forge/miniforge#download), also shown below
+
+```bash
+curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
+bash Miniforge3-$(uname)-$(uname -m).sh
+```
 
 Make sure to run `mamba init` and restart your terminal before proceeding.
 
+Run `mamba activate` if not already in the base environment
+
 Then, run:
 
 ```bash

scripts/install_dex_teleop.sh

@@ -1,6 +1,6 @@
 #!/bin/bash
 
-pip install mediapipe
+pip install mediapipe webcam numba
 
 echo "Copy udev rule file for the Logitech Webcam C930e."
 echo "sudo cp ./99-hello-dex-teleop-camera.rules /etc/udev/rules.d/"

src/setup.py

@@ -72,6 +72,7 @@
         "hydra-core",
         "timm>1.0.0",
         "huggingface_hub[cli]>=0.24.7",
+        # For mobile clip
         "open-clip-torch>=2.32.0",
         "transformers>=4.50.0",
         "retry",
@@ -81,8 +82,6 @@
         "accelerate >= 1.5.0",
         "einops",
         "protobuf",
-        # Meta neural nets
-        "segment-anything",
         # Compression tools
         "pyliblzfse",
         "webp>=0.3.0",

src/stretch/agent/robot_agent.py

@@ -956,7 +956,6 @@ def _update_scene_graph(self):
             self.scene_graph.update(self.get_voxel_map().get_instances())
         # For debugging - TODO delete this code
         self.scene_graph.get_relationships(debug=False)
-        # self.robot._rerun.update_scene_graph(self.scene_graph, self.semantic_sensor)
 
     def get_scene_graph(self) -> SceneGraph:
         """Return scene graph, such as it is."""