Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
218 commits
Select commit Hold shift + click to select a range
63041b0
Add semnav-specific / hydra-related changes to stretch_ai
blakerbuchanan Nov 5, 2024
3706b02
commit changes used for in-home hardware experiments
SaumyaSaxena Jan 10, 2025
a105448
installation update
hello-peiqi Jan 16, 2025
daa6880
grapheqa
hello-peiqi Jan 16, 2025
0e2e7ae
grapheqa fix
hello-peiqi Jan 17, 2025
665ac75
update
hello-peiqi Jan 27, 2025
6d12fbf
restore rerun
hello-peiqi Jan 27, 2025
c9d93f6
restore branch
hello-peiqi Jan 27, 2025
eda19da
update lfd docs
hello-peiqi Jan 31, 2025
4e31af9
dynamem fix
hello-peiqi Jan 31, 2025
f30ddc9
revert unexpected bugs
hello-peiqi Jan 31, 2025
c9eda02
revert unexpected bugs
hello-peiqi Jan 31, 2025
b7218f0
Merge branch 'main' into hello-peiqi/grapheqa
hello-peiqi Jan 31, 2025
95234da
Add executable flag to ros2 bridge server script
hello-binit Feb 4, 2025
d676b97
Merge branch 'bugfix/executable-bridge' of https://github.com/hello-r…
hello-peiqi Feb 4, 2025
216ddbb
grapheqa push
hello-peiqi Feb 4, 2025
7b37896
Merge branch 'main' into hello-peiqi/grapheqa
hello-peiqi Feb 4, 2025
c9feeba
Merge branch 'hello-peiqi/grapheqa' of https://github.com/hello-robot…
hello-peiqi Feb 4, 2025
b58cd69
debug to make scene grpah building work
hello-peiqi Feb 4, 2025
4a7ea35
update
hello-peiqi Feb 6, 2025
12dbed8
update
hello-peiqi Feb 7, 2025
15c8cef
update
hello-peiqi Feb 7, 2025
8032c56
checkpoint
hello-peiqi Feb 9, 2025
a384867
update
hello-peiqi Feb 11, 2025
a17cc36
update save image
hello-peiqi Feb 12, 2025
232da6e
update save best image
hello-peiqi Feb 15, 2025
b55a649
add planner
hello-peiqi Feb 17, 2025
a18fe80
debug
hello-peiqi Feb 18, 2025
051c4b8
debug planner
hello-peiqi Feb 18, 2025
7511362
write custom question
hello-peiqi Feb 19, 2025
1850efe
add discord support
hello-peiqi Feb 20, 2025
f8676a6
add captioner
hello-peiqi Feb 21, 2025
6d51d32
add deepseek
cpaxton Feb 22, 2025
8888ed8
add qwen deepseek and make support a little cleaner and more
cpaxton Feb 22, 2025
99855dc
updates
cpaxton Feb 22, 2025
1bede03
added qwen quantization and some cleanup
cpaxton Feb 22, 2025
c788f2e
update and add bitsandbytes
cpaxton Feb 22, 2025
8dd3991
update
cpaxton Feb 22, 2025
435b479
update
hello-peiqi Feb 23, 2025
bc4fcbc
suggestions on deepseek update
hello-peiqi Feb 27, 2025
91a1ffa
Merge branch 'cpaxton/deepseek' of https://github.com/hello-robot/str…
hello-peiqi Feb 27, 2025
285b5e9
find better captioner
hello-peiqi Mar 2, 2025
db15156
update installation
hello-peiqi Mar 2, 2025
c6ba6c8
update installation
hello-peiqi Mar 2, 2025
95abd77
fix setup
hello-peiqi Mar 2, 2025
a4d54d3
fix setup
hello-peiqi Mar 3, 2025
db09627
improve running speed
hello-peiqi Mar 3, 2025
5bb2f4a
exploration update
hello-peiqi Mar 7, 2025
5fce257
fix installation
hello-peiqi Mar 10, 2025
2ee6a53
fix installation
hello-peiqi Mar 10, 2025
09f6d84
update scene graph merging
hello-peiqi Mar 12, 2025
45b4a1f
improve performance
hello-peiqi Mar 14, 2025
9f6bac8
minor update
hello-peiqi Mar 14, 2025
1f572ca
pass unit test
hello-peiqi Mar 14, 2025
812d4b8
udpate owlsam
hello-peiqi Mar 14, 2025
b5b19b5
Merge branch 'main' into hello-peiqi/grapheqa_merge
hello-peiqi Mar 14, 2025
14bab14
fix bug
hello-peiqi Mar 14, 2025
7fcd248
Merge branch 'hello-peiqi/grapheqa_merge' of https://github.com/hello…
hello-peiqi Mar 14, 2025
70167f8
update docs
hello-peiqi Mar 16, 2025
9c7f227
add captioners
hello-peiqi Mar 17, 2025
7c16cfb
update
hello-peiqi Mar 19, 2025
718df94
update
hello-peiqi Mar 19, 2025
23a74c5
switch to qwen
hello-peiqi Mar 19, 2025
80d542b
upgrade
hello-peiqi Mar 20, 2025
b165b0b
update
hello-peiqi Mar 25, 2025
3293404
update
hello-peiqi Mar 25, 2025
d8e67bc
update
hello-peiqi Mar 25, 2025
c001ef1
update discord
hello-peiqi Mar 26, 2025
2b1bfa0
fix bug
hello-peiqi Mar 26, 2025
c18a121
minor update
hello-peiqi Mar 28, 2025
0a683c7
minor update in prompt
hello-peiqi Mar 28, 2025
33b229b
Stretch EQA (#263)
hello-peiqi May 2, 2025
c20b982
Merge branch 'main' into hello-peiqi/grapheqa_merge
hello-peiqi May 2, 2025
b1550bb
remove projects folder
hello-peiqi May 2, 2025
7d55bf6
add docs
hello-peiqi May 7, 2025
b27ec84
add video
hello-peiqi May 7, 2025
c6801ff
downsample image
hello-peiqi May 7, 2025
845f15f
small update
hello-peiqi May 7, 2025
d88bad3
switch to large image
hello-peiqi May 7, 2025
62f2272
minor update to documentation
hello-ck May 7, 2025
cae3e48
documentation update
hello-ck May 7, 2025
d342d98
more documentation edits
hello-ck May 7, 2025
a369db7
some docs edits
hello-peiqi May 7, 2025
96c8e4b
documentation edits
hello-ck May 7, 2025
c0d0118
resolve conflict
hello-peiqi May 7, 2025
d7d4e0c
Merge branch 'hello-peiqi/grapheqa_merge' of https://github.com/hello…
hello-peiqi May 7, 2025
4d10072
doc update
hello-peiqi May 7, 2025
0a63963
update docs
hello-peiqi May 9, 2025
0fff6db
update docs
hello-peiqi May 9, 2025
5e6b255
save docstring
hello-peiqi May 9, 2025
a318134
add docsstring
hello-peiqi May 9, 2025
ec7cd06
add docsstring
hello-peiqi May 9, 2025
e995d6a
add docsstring
hello-peiqi May 9, 2025
7b231f0
Hello peiqi/cleanup (#267)
hello-peiqi May 12, 2025
209d04a
update? (#268)
hello-peiqi May 15, 2025
04d8c7b
update
hello-peiqi May 17, 2025
6b5c95c
update
hello-peiqi May 17, 2025
97af229
update
hello-peiqi May 17, 2025
fa76fa3
update
hello-peiqi May 17, 2025
350f43c
update
hello-peiqi May 17, 2025
cb7b38e
update
hello-peiqi May 18, 2025
533c2f9
update
hello-peiqi May 18, 2025
75fd6ed
update
hello-peiqi May 18, 2025
13b9583
update
hello-peiqi May 20, 2025
47bc45e
udpate
hello-peiqi May 21, 2025
ec0dace
Hello peiqi/dynamem cpu (#270)
hello-peiqi Jun 6, 2025
1adc58d
fix bug
hello-peiqi Jun 8, 2025
1ff78d1
update
hello-peiqi Jun 10, 2025
542c895
update hyperparameter
hello-peiqi Jun 10, 2025
87dfda3
update hyperparameter
hello-peiqi Jun 10, 2025
54ddaf7
update hyperparameter
hello-peiqi Jun 10, 2025
97db994
update
hello-peiqi Jun 10, 2025
ee102a0
update
hello-peiqi Jun 10, 2025
92857a2
update gemini
hello-peiqi Jun 10, 2025
7023980
update
hello-peiqi Jun 12, 2025
a330ded
Hello peiqi/cvpr update (#274)
hello-peiqi Jun 15, 2025
82a59da
add SAMv2 installation
hello-peiqi Jun 17, 2025
9fd1972
comment small update
hello-peiqi Jun 17, 2025
8239dc2
remove moonlight
hello-peiqi Jun 17, 2025
c44030c
remove dinobot
hello-peiqi Jun 17, 2025
fd9cf9e
fix bug
hello-peiqi Jun 17, 2025
6038980
Merge branch 'hello-peiqi/grapheqa_merge' of https://github.com/hello…
hello-peiqi Jun 17, 2025
6d6a6e3
remove yolo
hello-peiqi Jun 18, 2025
e1c515e
remove detic
hello-peiqi Jun 18, 2025
b00d305
update installation
hello-peiqi Jun 19, 2025
a618b84
remove the remains of Detic
hello-peiqi Jun 20, 2025
217f49b
remove paligemma
hello-peiqi Jun 20, 2025
7e5b6c8
remove molmo
hello-peiqi Jun 20, 2025
2279b93
Hello peiqi/remove online audio (#278)
hello-peiqi Jun 22, 2025
9bd38aa
add terminate on failure
hello-peiqi Jun 23, 2025
2825f3c
Merge branch 'hello-peiqi/perception_cleaning' of https://github.com/…
hello-peiqi Jun 23, 2025
c538250
Hello peiqi/clean app (#279)
hello-peiqi Jun 24, 2025
91454d6
remove vlm planning
hello-peiqi Jun 24, 2025
d268083
replace git with blip
hello-peiqi Jun 24, 2025
781a6a2
remove dinov2siglip
hello-peiqi Jun 24, 2025
1518780
remove readme
hello-peiqi Jun 25, 2025
8996a64
remove unnecessary core
hello-peiqi Jun 26, 2025
0eb68f7
restore scripts
hello-peiqi Jun 26, 2025
4ed7def
fix installation
hello-peiqi Jun 26, 2025
b2eda9a
resolve conflict
hello-peiqi Jun 26, 2025
18512e9
update docs
hello-peiqi Jun 26, 2025
e315f8e
remove files
hello-peiqi Jul 2, 2025
051cf19
restore
hello-peiqi Jul 2, 2025
5609e73
remove navigation utils
hello-peiqi Jul 2, 2025
d758a9f
remove navigation app
hello-peiqi Jul 2, 2025
a2d7149
remove drivers
hello-peiqi Jul 2, 2025
255c3f5
fix orbslam3 bugs
hello-peiqi Jul 3, 2025
105e42c
Merge branch 'hello-peiqi/slam_checking' of https://github.com/hello-…
hello-peiqi Jul 3, 2025
f7a74ad
fix orbslam3 bugs
hello-peiqi Jul 3, 2025
04ba6bc
remove log
hello-peiqi Jul 4, 2025
d111536
remove unnecessary ros2 files
hello-peiqi Jul 4, 2025
9f83072
further clean
hello-peiqi Jul 4, 2025
9a4f92d
remove mobile sam
hello-peiqi Jul 5, 2025
9898cc8
deal with sam
hello-peiqi Jul 5, 2025
4cc76c9
Merge branch 'hello-peiqi/devel' of https://github.com/hello-robot/st…
hello-peiqi Jul 5, 2025
8408ef8
fix conflict
hello-peiqi Jul 5, 2025
12eabd6
update docs
hello-peiqi Jul 5, 2025
463fdeb
update slam
hello-peiqi Jul 5, 2025
5daf4b9
Merge branch 'hello-peiqi/devel' of https://github.com/hello-robot/st…
hello-peiqi Jul 5, 2025
4d9a3c7
checkpoint
hello-peiqi Jul 5, 2025
2cafd61
update install.sh
hello-peiqi Jul 7, 2025
7f2a24a
fix conflist
hello-peiqi Jul 7, 2025
046977c
remove map loader app
hello-peiqi Jul 7, 2025
6632742
remove more apps
hello-peiqi Jul 7, 2025
5c1fab0
remove load map util
hello-peiqi Jul 7, 2025
9d852f1
remove demo
hello-peiqi Jul 7, 2025
15cad73
remove ENV and Agent
hello-peiqi Jul 7, 2025
f7fcb07
resolve conflict
hello-peiqi Jul 7, 2025
bb4fd1e
Merge branch 'hello-peiqi/slam_checking' of https://github.com/hello-…
hello-peiqi Jul 7, 2025
2f51079
clean core
hello-peiqi Jul 7, 2025
e9734dd
clean drivers
hello-peiqi Jul 7, 2025
b071207
remove orbslam docs
hello-peiqi Jul 7, 2025
e36b10e
Merge branch 'hello-peiqi/devel' of https://github.com/hello-robot/st…
hello-peiqi Jul 7, 2025
b3ecafc
update detic
hello-peiqi Jul 7, 2025
06c7f58
update detic
hello-peiqi Jul 7, 2025
010acfa
fix conflict
hello-peiqi Jul 7, 2025
a255da4
unify agent
hello-peiqi Jul 7, 2025
aaad310
Merge branch 'hello-peiqi/devel' of https://github.com/hello-robot/st…
hello-peiqi Jul 8, 2025
2afb52e
clean robot agent
hello-peiqi Jul 9, 2025
61b2e48
remove get detections
hello-peiqi Jul 9, 2025
2a6cf16
remove unused attr
hello-peiqi Jul 9, 2025
6e5c12d
clean more files
hello-peiqi Jul 9, 2025
d8b7881
fix bug
hello-peiqi Jul 9, 2025
df97315
clean slam
hello-peiqi Jul 9, 2025
bf4c9aa
fix conflict
hello-peiqi Jul 9, 2025
616eade
fix ai pick up bug
hello-peiqi Jul 9, 2025
a89da71
Merge branch 'hello-peiqi/devel' of https://github.com/hello-robot/st…
hello-peiqi Jul 9, 2025
0753e6e
update
hello-peiqi Jul 9, 2025
06f5a56
fix bug
hello-peiqi Jul 9, 2025
7933955
fix typo
hello-peiqi Jul 9, 2025
3a6a2cf
fix match method
hello-peiqi Jul 10, 2025
bd11f17
fix bug
hello-peiqi Jul 11, 2025
850f01c
installation
hello-peiqi Jul 11, 2025
0cb41a8
Hello peiqi/fix simulation (#283)
hello-peiqi Jul 16, 2025
493b50b
tune readme
hello-peiqi Jul 16, 2025
fd72bb9
Merge branch 'hello-peiqi/devel' of https://github.com/hello-robot/st…
hello-peiqi Jul 16, 2025
2651cbd
clean things
hello-peiqi Jul 17, 2025
1a1bd2a
minor installation update
hello-peiqi Jul 18, 2025
2fca477
remove lfd files
hello-peiqi Jul 20, 2025
9b3efe7
Hello peiqi/slam checking (#281)
hello-peiqi Jul 23, 2025
03b8f9d
debug grasp
hello-peiqi Jul 23, 2025
704591b
slight tunning
hello-peiqi Jul 23, 2025
4d1cf8d
fix conflict
hello-peiqi Jul 23, 2025
8b33946
remove more files
hello-peiqi Jul 23, 2025
4b5d76a
update docs
hello-peiqi Jul 29, 2025
fd154e9
clean ik solver
hello-peiqi Jul 29, 2025
fedbafc
clean ik solver
hello-peiqi Jul 29, 2025
1dcdb80
cleaning
hello-peiqi Jul 29, 2025
809dfae
cleaning
hello-peiqi Jul 29, 2025
bce301f
fix docs
hello-peiqi Jul 29, 2025
2a7c272
restore dynamem_task
hello-peiqi Jul 29, 2025
d589003
Merge branch 'hello-peiqi/lfd' of https://github.com/hello-robot/stre…
hello-peiqi Jul 29, 2025
5f69176
update
hello-peiqi Jul 30, 2025
0ca16ab
fix conflict
hello-peiqi Jul 30, 2025
c6b11c6
clean codes
hello-peiqi Jul 30, 2025
209356c
add rum format file
hello-peiqi Aug 2, 2025
a06b65a
update docs
hello-peiqi Aug 2, 2025
0c07cee
Added a merge commit to resolve conflicts
hello-robot-shehab Oct 9, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ We recommend the following hardware to run Stretch AI. Other GPUs and other vers

Artificial intelligence (AI) for robots often has complex dependencies, including the need for trained models. Consequently, installing *stretch-ai* from source can be challenging.

First, you will need to install software on your Stretch robot and another computer with a GPU (*GPU computer*). Use the following link to go to the installation instructions: [Instructions for Installing Stretch AI](https://github.com/hello-robot/stretch_ai/blob/main/docs/start_with_docker_plus_virtenv.md)
First, you will need to install software on your Stretch robot and another computer with a GPU (*GPU computer*). Use the following link to go to the installation instructions: [Instructions for Installing Stretch AI](./docs/start_with_docker_plus_virtenv.md)

Once you've completed this installation, you can start the server on your Stretch robot. Prior to running the script, you need to have homed your robot with `stretch_robot_home.py`. Then, run the following command:

Expand Down
4 changes: 2 additions & 2 deletions docker/Dockerfile.ros2
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,8 @@ USER hello-robot
RUN echo "Installing Stretch AI package"
RUN pip install --upgrade pip setuptools packaging build meson ninja
# Do this if you want to check out a specific branch; it's important to make sure versions match:
# RUN git clone https://github.com/hello-robot/stretch_ai
RUN git clone https://github.com/hello-robot/stretch_ai --branch hello-peiqi/slam_checking
# RUN git clone https://github.com/hello-robot/stretch_ai --branch main
RUN git clone https://github.com/hello-robot/stretch_ai
WORKDIR /home/hello-robot/stretch_ai
RUN pip install -e src/

Expand Down
37 changes: 0 additions & 37 deletions docs/arucos.md

This file was deleted.

3 changes: 0 additions & 3 deletions docs/data_collection.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,9 +97,6 @@ Launch this command from the directory where URDFs are stored (default is root o
If you have already ran an app with `--robot-ip` flag, such as the `view_images` test app, you can omit the flag in subsequent runs.

```bash
# The -s flag enables png images to be saved in addition to videos, which is faster for model training if training is CPU bound (no video decoding)

TASK_NAME=<name-of-task>
python3 -m stretch.app.dex_teleop.ros2_leader --robot_ip $ROBOT_IP --task-name $TASK_NAME --teleop-mode <teleop-mode> --clutch
```

Expand Down
10 changes: 0 additions & 10 deletions docs/install_details.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,16 +113,6 @@ cd ~/ament_ws
colcon build --packages-select stretch_ros2_bridge
```

#### Experimental: Install ORB-SLAM3 On the Robot (Advanced)

[ORB-SLAM3](https://arxiv.org/pdf/2007.11898) is an open-source VSLAM (visual slam) library. Using it in conjunction with LIDAR-based localization can improve performance in many environments. Installation is documented in a [separate file](docs/orbslam3.md).

*Installation is not required to use Stretch AI.* If you chose to do so, you can then then use the ORB-SLAM3 version of the server launch file:

```
ros2 launch stretch_ros2_bridge server_orbslam3.launch.py
```

### Using LLMs

We use many open-source LLMs from [Huggingface](https://huggingface.co/). TO use them, you will need to make sure `transformers` is installed and up to date. You can install it with:
Expand Down
4 changes: 3 additions & 1 deletion docs/learning_from_demonstration.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,9 @@ Our LfD code is a [fork of LeRobot](https://github.com/hello-robot/lerobot). The

## Format data and push to huggingface repo

### [Authenticate with huggingface-cli](https://huggingface.co/docs/huggingface_hub/en/guides/cli)
### [Authenticate with huggingface-cli](https://huggingface.co/docs/huggingface_hub/en/guides/cli)

You should finish this before doing the remaining steps or you might waste 1 hour in uploading dataset to huggingface.

### Optional: login to weights and biases (wandb)

Expand Down
11 changes: 10 additions & 1 deletion docs/start_with_docker_plus_virtenv.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,10 +136,19 @@ First, check to see if mamba is installed on your computer:
mamba
```

If you get an "command not found" error, then follow the install instructions for mamba here: https://github.com/conda-forge/miniforge#download
If you get an "command not found" error, then you have not yet installed mamba.

Follow the install instructions for mamba [here](https://github.com/conda-forge/miniforge#download), also shown below

```bash
curl -L -O "https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-$(uname)-$(uname -m).sh"
bash Miniforge3-$(uname)-$(uname -m).sh
```

Make sure to run `mamba init` and restart your terminal before proceeding.

Run `mamba activate` if not already in the base environment

Then, run:

```bash
Expand Down
2 changes: 1 addition & 1 deletion scripts/install_dex_teleop.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/bin/bash

pip install mediapipe
pip install mediapipe webcam numba

echo "Copy udev rule file for the Logitech Webcam C930e."
echo "sudo cp ./99-hello-dex-teleop-camera.rules /etc/udev/rules.d/"
Expand Down
3 changes: 1 addition & 2 deletions src/setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@
"hydra-core",
"timm>1.0.0",
"huggingface_hub[cli]>=0.24.7",
# For mobile clip
"open-clip-torch>=2.32.0",
"transformers>=4.50.0",
"retry",
Expand All @@ -81,8 +82,6 @@
"accelerate >= 1.5.0",
"einops",
"protobuf",
# Meta neural nets
"segment-anything",
# Compression tools
"pyliblzfse",
"webp>=0.3.0",
Expand Down
1 change: 0 additions & 1 deletion src/stretch/agent/robot_agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -956,7 +956,6 @@ def _update_scene_graph(self):
self.scene_graph.update(self.get_voxel_map().get_instances())
# For debugging - TODO delete this code
self.scene_graph.get_relationships(debug=False)
# self.robot._rerun.update_scene_graph(self.scene_graph, self.semantic_sensor)

def get_scene_graph(self) -> SceneGraph:
"""Return scene graph, such as it is."""
Expand Down
233 changes: 0 additions & 233 deletions src/stretch/app/dex_teleop/dex_teleop.py

This file was deleted.

Loading
Loading