All Remaining MPI Examples have been Added

Author: Thomas Weise

.gitignore

@@ -158,7 +158,36 @@ hs_err_pid*
 
 
 ## MPI
-### Linux Builds of C examples
+## Linux Builds of C examples
 mpi/bareBones
+mpi/basicInfo
+mpi/simplePointToPoint1
+mpi/simplePointToPoint2
+mpi/piPointToPoint
+mpi/deadlock
+mpi/nonBlockingPointToPoint
+mpi/piNonBlockingPointToPoint
+mpi/broadcast
+mpi/gatherScatterBareBones
+mpi/gatherScatterPrimes
+mpi/piGatherScatter
+mpi/reducePrimes
+mpi/structTest
+mpi/structScatter
 ## Windows Builds of C examples
-mpi/bareBones.exe
+mpi/bareBones.exe
+mpi/basicInfo.exe
+mpi/simplePointToPoint1.exe
+mpi/simplePointToPoint2.exe
+mpi/piPointToPoint.exe
+mpi/simplePtP2.exe
+mpi/deadlock.exe
+mpi/nonBlockingPointToPoint.exe
+mpi/piNonBlockingPointToPoint.exe
+mpi/broadcast.exe
+mpi/gatherScatterBareBones.exe
+mpi/gatherScatterPrimes.exe
+mpi/piGatherScatter.exe
+mpi/reducePrimes.exe
+mpi/structTest.exe
+mpi/structScatter.exe

mpi/.gitignore

@@ -61,5 +61,34 @@ hs_err_pid*
 ## MPI
 ## Linux Builds of C examples
 bareBones
+basicInfo
+simplePointToPoint1
+simplePointToPoint2
+piPointToPoint
+deadlock
+nonBlockingPointToPoint
+piNonBlockingPointToPoint
+broadcast
+gatherScatterBareBones
+gatherScatterPrimes
+piGatherScatter
+reducePrimes
+structTest
+structScatter
 ## Windows Builds of C examples
-bareBones.exe
+bareBones.exe
+basicInfo.exe
+simplePointToPoint1.exe
+simplePointToPoint2.exe
+piPointToPoint.exe
+simplePtP2.exe
+deadlock.exe
+nonBlockingPointToPoint.exe
+piNonBlockingPointToPoint.exe
+broadcast.exe
+gatherScatterBareBones.exe
+gatherScatterPrimes.exe
+piGatherScatter.exe
+reducePrimes.exe
+structTest.exe
+structScatter.exe

mpi/README.md

@@ -8,6 +8,90 @@ The following examples are included in this folder.
 
 ## 1.1. Bare Bones
 
-A simple MPI test program which does nothing except of initializing and disposing the MPI sub system.
+A simple MPI test program which does nothing except of initializing and disposing the MPI sub system. Launch 1 instance.
 
-1. [bareBones.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/bareBones.c)
+1. [bareBones.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/bareBones.c)
+
+## 1.2. Basic Info
+
+A simple MPI test program which does nothing except of initializing and disposing the MPI sub system and printing the size of the current communicator and the rank of the current process in it. Launch any number of instances.
+
+1. [basicInfo.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/basicInfo.c)
+
+## 1.3. Simple Point-to-Point Communication
+
+A simple MPI program which performs some simple point-to-point communication: Each process with an even rank sends a message to the process with the next-higher rank and wants to receive a message from the process with the next-lower rank. For the odd-ranked processes, it is the other way around. Launch 2 instances, or 2n instances.
+
+1. [simplePointToPoint.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/simplePointToPoint.c)
+
+## 1.4. Simple Point-to-Point Communication 2
+
+A simple MPI program which performs some simple point-to-point communication: The process with rank 0 sends a string to the process with rank 1 who receives it. Launch two instances.
+
+1. [simplePointToPoint2.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/simplePointToPoint2.c)
+
+## 1.5. Estimate Pi with Point-to-Point Communication
+
+This program tries to estimate Pi in the same way as done in our Java [client](http://github.com/thomasWeise/distributedComputingExamples/tree/master/sockets/java/src/PiClient.java)/[server](http://github.com/thomasWeise/distributedComputingExamples/tree/master/sockets/java/src/PiServer.java) example for [sockets](http://github.com/thomasWeise/distributedComputingExamples/tree/master/sockets/java/) - just with MPI. Launch 4 or 5 instances. See also examples 1.8 and 1.12.
+
+1. [piPointToPoint.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/piPointToPoint.c)
+
+## 1.6. Deadlock Error
+
+This program compiles but will enter a deadlock if you run it. The reason is that the processes wait for each other in a cycle. Launch two instances to see how they hang.
+
+1. [deadlock.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/deadlock.c)
+
+## 1.7. Non-Blocking Point-to-Point Communication
+
+This program is very similar to the previous one which caused a deadlock. However, we now use non-blocking point-to-point communication. This means that we can initiate a message receive action and then send a message and then wait for the receive to complete. The deadlock disappears. Launch 2 instances.
+
+1. [nonBlockingPointToPoint.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/nonBlockingPointToPoint.c)
+
+## 1.8. Estimate Pi with Non-Blocking Point-to-Point Communication
+
+Like example 1.5, we try to estimate Pi with point-to-point communication. However, now we perform an asynchronous computation and use non-block point-to-point communication. Launch 4 or 5 instances. See also examples 1.5 and 1.12.
+
+1. [piNonBlockingPointToPoint.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/piNonBlockingPointToPoint.c)
+
+## 1.9. Broadcast
+
+The root node will broadcast a message to everyone. Launch 5 istances.
+
+1. [broadcast.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/broadcast.c)
+
+## 1.10. Gather-Scatter: The Bare Bones
+
+This example shows the bare bones of a gather-scatter based communication.
+
+1. [gatherScatterBareBones.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/gatherScatterBareBones.c)
+
+## 1.11. Gather-Scatter: Count Primes
+
+We use a gather-scatter based communication to count the prime numbers amongst the first 1024 numbers. The number range is divided among all workers. See also example 14, launch 4 instances.
+
+1. [gatherScatterPrimes.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/gatherScatterPrimes.c)
+
+## 1.12. Gather-Scatter: Estimate Pi
+
+This example again tries to estimate Pi, but this time we use gather-scatter based communication. Launch 4 or 5 instances. See also examples 1.5 and 1.8.
+
+1. [piGatherScatter.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/piGatherScatter.c)
+
+## 1.13. Reduce: Count Primes
+
+Like in example 1.11, we want to count the number of primes amongst the first 1024 natural numbers. This time we use `reduce` in the communication. Launch 4 instances
+
+1. [reducePrimes.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/reducePrimes.c)
+
+## 1.14. Memory Layout of a Struct
+
+This example does no communication at all, but it prints the memory layout of a `struct`. This shows that the compiler may align fields in many ways and we cannot compute on where a field of a `struct` but need to use proper addressing. 
+
+1. [structTest.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/structTest.c)
+
+## 1.15. Struct with Scatter
+
+We define a `struct` datatype for MPI and then send such `struct`s via scatter. Launch 4 instances.  
+
+1. [structScatter.c](http://github.com/thomasWeise/distributedComputingExamples/tree/master/mpi/structScatter.c)

mpi/basicInfo.c

@@ -0,0 +1,20 @@
+#include <mpi.h>
+#include <stdio.h>
+
+int main(int argc, char **argv) {
+  int size, rank;
+
+  MPI_Init(&argc, &argv);
+
+  MPI_Comm_size(MPI_COMM_WORLD, &size);
+  MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+
+  if(rank == 0) {
+    printf("Hi from Master\n");
+  } else {
+    printf("Just Slave %d out of %d\n", rank, size);
+  }
+
+  MPI_Finalize();
+  return 0;
+}

mpi/broadcast.c

@@ -0,0 +1,22 @@
+#include <mpi.h>
+#include <stdio.h>
+
+int main(int argc, char *argv[]) {
+  char message[60];
+  int rank, size, root;
+  MPI_Status status;
+
+  MPI_Init(&argc, &argv);
+  MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+  MPI_Comm_size(MPI_COMM_WORLD, &size);
+
+  if (rank == 0) {
+    sprintf(message, "Message from root (rank %d)", rank);
+  }
+
+  MPI_Bcast(message, 60, MPI_CHAR, root, MPI_COMM_WORLD);
+  printf("The message sent/received at node %d is \"%s\"\n", rank, message);
+
+  MPI_Finalize();
+  return 0;
+}

mpi/deadlock.c

@@ -0,0 +1,25 @@
+#include <stdio.h>
+#include <string.h>
+#include <mpi.h>
+
+int main(int argc, char **argv) {
+  int         rank, size, prev, next;
+  MPI_Status  status;
+  char        message[20];
+
+  MPI_Init(&argc, &argv);
+  MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+  MPI_Comm_size(MPI_COMM_WORLD, &size);
+
+  prev = ((size + rank - 1) % size);
+  next = ((rank + 1) % size);
+  strcpy(message, "Important message!");
+
+  MPI_Recv(message, 20, MPI_CHAR, prev, 0, MPI_COMM_WORLD, &status);
+  printf("Process %d received message %s from process %d.\n", rank, message, prev);
+  printf("Process %d is sending message %s to process %d.\n", rank, message, next);
+  MPI_Send(message, 20, MPI_CHAR, next, 0, MPI_COMM_WORLD);
+
+  MPI_Finalize();
+  return 0;
+}

mpi/gatherScatterBareBones.c

@@ -0,0 +1,31 @@
+#include <mpi.h>
+#include <stdio.h>
+#include <math.h>
+
+#define DATA_SIZE 1024
+
+int main(int argc, char *argv[]) {
+  int send[DATA_SIZE], recv[DATA_SIZE];
+  int rank, size, count, root, res;
+  MPI_Status status;
+
+  MPI_Init(&argc, &argv);
+  MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+  MPI_Comm_size(MPI_COMM_WORLD, &size);
+
+  if(rank == 0) { //If root: Generate data to be distributed.
+  }
+
+  //Send data to all nodes. here: an integer array of length "count".
+  count = (DATA_SIZE / size);
+  MPI_Scatter(send, count, MPI_INT, recv, count, MPI_INT, 0, MPI_COMM_WORLD);
+
+  // Each node now processes its share of data and sends the results (here: int "res") to root.
+  MPI_Gather(&res, 1, MPI_INT, recv, 1, MPI_INT, 0, MPI_COMM_WORLD);
+
+  if(rank == 0) { //If root: process the received data.
+  }
+
+  MPI_Finalize();
+  return 0;
+}

mpi/gatherScatterPrimes.c

@@ -0,0 +1,46 @@
+#include <mpi.h>
+#include <stdio.h>
+#include <math.h>
+
+#define DATA_SIZE 1024
+
+int main(int argc, char *argv[]) {
+  int send[DATA_SIZE], recv[DATA_SIZE];
+  int rank, size, count, root, res, i, j;
+  MPI_Status status;
+
+  MPI_Init(&argc, &argv);
+  MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+  MPI_Comm_size(MPI_COMM_WORLD, &size);
+
+  if(rank == 0) { //generate data if root
+    for(i = DATA_SIZE; (--i)>=0; ) { send[i] = (i + 1); }
+  }
+
+  count = (DATA_SIZE / size);
+  MPI_Scatter(send, count, MPI_INT, recv, count, MPI_INT, 0, MPI_COMM_WORLD);
+
+  // each node now processes its share of data
+  res = count; //here: count how many prime numbers are contained in the array
+  outer: for(i = count; (--i) >= 0; ) {
+    for(j = ((int)(sqrt(recv[i]))|1); j>1; j--) {
+      if((recv[i] % j) == 0) {
+        res--;
+        break; }
+    }
+  }
+  printf("Process %d discovered %d primes in the numbers from %d to %d.\n", rank, res, recv[0], recv[count-1]);
+
+  MPI_Gather(&res, 1, MPI_INT, recv, 1, MPI_INT, 0, MPI_COMM_WORLD);
+
+  if(rank == 0) { //if root, process the received data
+    res = 0;
+    for(i = size; (--i) >= 0; ) { //add up the prime number counts
+      res += recv[i];
+    }
+    printf("The total number of primes in the first %d natural numbers is %d.\n", (count*size), res);
+  }
+
+  MPI_Finalize();
+  return 0;
+}

mpi/make_linux.sh

@@ -9,7 +9,35 @@ set -o errexit   # set -e : exit the script if any statement returns a non-true
 echo "Building MPI examples in C for Linux."
 
 rm -f bareBones
+rm -f basicInfo
+rm -f simplePointToPoint
+rm -f simplePointToPoint2
+rm -f piPointToPoint
+rm -f deadlock
+rm -f nonBlockingPointToPoint
+rm -f piNonBlockingPointToPoint
+rm -f broadcast
+rm -f gatherScatterBareBones
+rm -f gatherScatterPrimes
+rm -f piGatherScatter
+rm -f reducePrimes
+rm -f structTest
+rm -f structScatter
 
 mpicc bareBones.c -o bareBones
+mpicc basicInfo.c -o basicInfo
+mpicc simplePointToPoint1.c -o simplePointToPoint1
+mpicc simplePointToPoint2.c -o simplePointToPoint2
+mpicc piPointToPoint.c -o piPointToPoint
+mpicc deadlock.c -o deadlock
+mpicc nonBlockingPointToPoint.c -o nonBlockingPointToPoint
+mpicc piNonBlockingPointToPoint.c -o piNonBlockingPointToPoint
+mpicc broadcast.c -o broadcast
+mpicc gatherScatterBareBones.c -o gatherScatterBareBones
+mpicc gatherScatterPrimes.c -o gatherScatterPrimes -lm
+mpicc piGatherScatter.c -o piGatherScatter
+mpicc reducePrimes.c -o reducePrimes -lm
+mpicc structTest.c -o structTest
+mpicc structScatter.c -o structScatter
 
 echo "Finished building MPI examples in C for Linux."

mpi/nonBlockingPointToPoint.c

@@ -0,0 +1,29 @@
+#include <mpi.h>
+#include <stdio.h>
+
+int main(int argc, char *argv[]) {
+  int           rank, size, prev, next;
+  char          buffer[30], buffer2[30];
+  MPI_Request   request, request2;
+  MPI_Status    status;
+
+  MPI_Init(&argc,&argv);
+  MPI_Comm_size(MPI_COMM_WORLD, &size);
+  MPI_Comm_rank(MPI_COMM_WORLD, &rank);
+
+  next = ((rank + 1) % size);
+  prev = ((rank + size - 1) % size);
+
+  MPI_Irecv(buffer, 30, MPI_CHAR, prev, 42, MPI_COMM_WORLD, &request);
+
+  sprintf(buffer2, "Non-blocking from %d!", rank);
+  MPI_Isend(buffer2, 30, MPI_CHAR, next, 42, MPI_COMM_WORLD, &request2);
+
+  MPI_Wait(&request, &status);
+  printf("%d received \"%s\"\n", rank, buffer);
+
+  MPI_Wait(&request2, &status);
+
+  MPI_Finalize();
+  return 0;
+}