Skip to content

Commit a15974c

Browse files
SyntaxRulesSyntaxRules
authored andcommitted
fix(post): parallel shells
1 parent bc46751 commit a15974c

1 file changed

Lines changed: 10 additions & 0 deletions

File tree

_posts/2017-1-3-parallel-shell.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@ tags: hpc software utilities
1010
1111

1212
There are 3 popular ways to perform parallel actions on HPC clusters:
13+
1314
- Using a shell script
1415
- Using Pdsh
1516
- Using ClusterShell
@@ -22,6 +23,7 @@ Before we dive into these 3 methods, its important to note that you should have
2223
This method can be done with any scripting language, but I'll use bash.
2324

2425
Create a script that you want ran:
26+
2527
```
2628
#!/bin/bash
2729
@@ -43,6 +45,7 @@ And run it with `sh your-script.sh`. This will run your command on all nodes, on
4345
> [Pdsh Github](https://github.com/grondo/pdsh)
4446
4547
Using pdsh is simple enough:
48+
4649
```
4750
# perform th uptime command on node1, node2, node3 and node 4
4851
$ pdsh -w node[1-4] uptime
@@ -56,16 +59,23 @@ node4: 19:20:45 up 1 day, 19:19, 1 user, load average: 0.29, 0.57, 0.53
5659
## Using ClusterShell
5760

5861
ClusterShell is built and maintained by CEA-HPC and claims to be effective on supercomputers with 5,000 compute nodes. Using ClusterShell is very similar to pdsh
62+
5963
```
6064
$ clush -w foo[1-5] echo "Hello World"
6165
```
66+
6267
ClusterShell does some merging of outputs and does a better job of handling hanging nodes than pdsh.
6368

6469
## References
6570

6671
[Bash for loop Syntax](https://www.cyberciti.biz/faq/bash-for-loop/)
72+
6773
[Pdsh github](https://github.com/grondo/pdsh)
74+
6875
[Pdsh man page](https://linux.die.net/man/1/pdsh)
76+
6977
[Pdsh vs ClusterShell](https://github.com/cea-hpc/clustershell/wiki/Pdsh)
78+
7079
[ClusterShell Github](https://github.com/cea-hpc/clustershell)
80+
7181
[ClusterShell main page](http://cea-hpc.github.io/clustershell/)

0 commit comments

Comments
 (0)