Tutorials: integration-batch-21 (5 tutorials)

tuts/116-aws-xray-gs/README.md

@@ -0,0 +1,56 @@
+# X-Ray: Send traces and query them
+
+Send trace segments to AWS X-Ray, query trace summaries, retrieve full traces, create a trace group, and view the service graph.
+
+## Source
+
+https://docs.aws.amazon.com/xray/latest/devguide/xray-api-sendingdata.html
+
+## Use case
+
+- **ID**: xray/getting-started
+- **Level**: intermediate
+- **Core actions**: `xray:PutTraceSegments`, `xray:GetTraceSummaries`, `xray:BatchGetTraces`, `xray:CreateGroup`
+
+## Steps
+
+1. Send a trace segment and subsegment
+2. Get trace summaries
+3. Get full trace details
+4. Create a trace group with a filter expression
+5. Get the service graph
+
+## Resources created
+
+| Resource | Type |
+|----------|------|
+| `tut-group-<random>` | X-Ray group |
+
+## Duration
+
+~10 seconds
+
+## Cost
+
+No charge. X-Ray provides a free tier of 100,000 traces recorded and 1,000,000 traces scanned per month.
+
+## Related docs
+
+- [Sending trace data to X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-api-sendingdata.html)
+- [Retrieving trace data](https://docs.aws.amazon.com/xray/latest/devguide/xray-api-gettingdata.html)
+- [Using groups in X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-console-groups.html)
+- [AWS X-Ray pricing](https://aws.amazon.com/xray/pricing/)
+
+---
+
+## Appendix
+
+| Field | Value |
+|-------|-------|
+| Date | 2026-04-14 |
+| Script lines | 87 |
+| Exit code | 0 |
+| Runtime | 10s |
+| Steps | 5 |
+| Issues | Fixed parent_id extraction quoting |
+| Version | v1 |

tuts/116-aws-xray-gs/REVISION-HISTORY.md

@@ -0,0 +1,8 @@
+# Revision History: 116-aws-xray-gs
+
+## Shell (CLI script)
+
+### 2026-04-14 v1 published
+- Type: functional
+- Initial version
+

tuts/116-aws-xray-gs/aws-xray-gs.md

@@ -0,0 +1,113 @@
+# Send traces and query them with AWS X-Ray
+
+## Overview
+
+In this tutorial, you use the AWS CLI to send trace segments to AWS X-Ray, query trace summaries, retrieve full trace details, create a trace group with a filter expression, and view the service graph. You then delete the group during cleanup.
+
+## Prerequisites
+
+- AWS CLI installed and configured with appropriate permissions.
+- Python 3 installed (used to generate trace IDs and timestamps).
+- An IAM principal with permissions for `xray:PutTraceSegments`, `xray:GetTraceSummaries`, `xray:BatchGetTraces`, `xray:CreateGroup`, `xray:DeleteGroup`, and `xray:GetServiceGraph`.
+
+## Step 1: Send a trace segment
+
+Generate a trace ID and send a segment representing an HTTP request, followed by a subsegment representing a downstream call.
+
+```bash
+TRACE_ID=$(python3 -c "import time,random;print(f'1-{int(time.time()):08x}-{random.randbytes(12).hex()}')")
+NOW=$(python3 -c "import time;print(time.time())")
+END=$(python3 -c "import time;print(time.time()+0.5)")
+
+SEGMENT="{\"trace_id\":\"$TRACE_ID\",\"id\":\"$(openssl rand -hex 8)\",\"name\":\"tutorial-service\",\"start_time\":$NOW,\"end_time\":$END,\"http\":{\"request\":{\"method\":\"GET\",\"url\":\"https://example.com/api/items\"},\"response\":{\"status\":200}}}"
+
+aws xray put-trace-segments --trace-segment-documents "$SEGMENT"
+```
+
+X-Ray trace IDs follow the format `1-<unix epoch hex>-<96-bit random hex>`. Each segment needs a unique 64-bit hex ID, a name, and start/end times as Unix epoch floats.
+
+Send a subsegment linked to the parent segment:
+
+```bash
+PARENT_ID=$(echo "$SEGMENT" | python3 -c "import sys,json;print(json.load(sys.stdin)['id'])")
+SUBSEG="{\"trace_id\":\"$TRACE_ID\",\"id\":\"$(openssl rand -hex 8)\",\"name\":\"database-query\",\"start_time\":$NOW,\"end_time\":$END,\"parent_id\":\"$PARENT_ID\"}"
+aws xray put-trace-segments --trace-segment-documents "$SUBSEG"
+```
+
+Subsegments use `parent_id` to link to their parent segment. This creates the hierarchy visible in the X-Ray trace map.
+
+## Step 2: Get trace summaries
+
+Query recent trace summaries. X-Ray takes a few seconds to index new traces.
+
+```bash
+START_TIME=$(python3 -c "import time;print(int(time.time()-60))")
+END_TIME=$(python3 -c "import time;print(int(time.time()))")
+
+aws xray get-trace-summaries \
+    --start-time "$START_TIME" --end-time "$END_TIME" \
+    --query 'TraceSummaries[:3].{TraceId:Id,Duration:Duration,Status:Http.HttpStatus,Method:Http.HttpMethod}' \
+    --output table
+```
+
+Trace summaries include the trace ID, duration, HTTP status, and response time. Use `--filter-expression` to narrow results by service name, status code, or annotation.
+
+## Step 3: Get full trace details
+
+Retrieve the complete trace including all segments and subsegments.
+
+```bash
+aws xray batch-get-traces --trace-ids "$TRACE_ID" \
+    --query 'Traces[0].Segments[].{Id:Id}' --output table
+```
+
+`batch-get-traces` returns the raw segment documents. You can retrieve up to 5 trace IDs per call.
+
+## Step 4: Create a trace group
+
+Create a group that filters traces by service name.
+
+```bash
+GROUP_NAME="tut-group-$(openssl rand -hex 4)"
+
+GROUP_ARN=$(aws xray create-group --group-name "$GROUP_NAME" \
+    --filter-expression 'service("tutorial-service")' \
+    --query 'Group.GroupARN' --output text)
+echo "Group ARN: $GROUP_ARN"
+```
+
+Groups let you organize traces by filter expression. Each group generates its own service graph and CloudWatch metrics.
+
+## Step 5: Get the service graph
+
+View the service graph for the time window containing your traces.
+
+```bash
+aws xray get-service-graph \
+    --start-time "$START_TIME" --end-time "$END_TIME" \
+    --query 'Services[].{Name:Name,Type:Type,Edges:Edges|length(@)}' \
+    --output table
+```
+
+The service graph shows services as nodes and their connections as edges. It may take a few seconds after sending traces for the graph to populate.
+
+## Cleanup
+
+Delete the trace group. Trace data itself expires automatically based on your X-Ray retention settings (default 30 days).
+
+```bash
+aws xray delete-group --group-arn "$GROUP_ARN"
+```
+
+The script automates all steps including cleanup:
+
+```bash
+bash aws-xray-gs.sh
+```
+
+## Related resources
+
+- [Sending trace data to X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-api-sendingdata.html)
+- [Retrieving trace data](https://docs.aws.amazon.com/xray/latest/devguide/xray-api-gettingdata.html)
+- [Using groups in X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-console-groups.html)
+- [AWS X-Ray pricing](https://aws.amazon.com/xray/pricing/)

tuts/116-aws-xray-gs/aws-xray-gs.sh

@@ -0,0 +1,88 @@
+#!/bin/bash
+# Tutorial: Send traces and query them with AWS X-Ray
+# Source: https://docs.aws.amazon.com/xray/latest/devguide/xray-api-sendingdata.html
+
+WORK_DIR=$(mktemp -d)
+LOG_FILE="$WORK_DIR/xray-$(date +%Y%m%d-%H%M%S).log"
+exec > >(tee -a "$LOG_FILE") 2>&1
+
+REGION=${AWS_DEFAULT_REGION:-${AWS_REGION:-$(aws configure get region 2>/dev/null)}}
+if [ -z "$REGION" ]; then
+    echo "ERROR: No AWS region configured. Set one with: export AWS_DEFAULT_REGION=us-east-1"
+    exit 1
+fi
+export AWS_DEFAULT_REGION="$REGION"
+echo "Region: $REGION"
+
+RANDOM_ID=$(cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 8 | head -n 1)
+GROUP_NAME="tut-group-${RANDOM_ID}"
+
+handle_error() { echo "ERROR on line $1"; trap - ERR; cleanup; exit 1; }
+trap 'handle_error $LINENO' ERR
+
+cleanup() {
+    echo ""
+    echo "Cleaning up resources..."
+    [ -n "$GROUP_ARN" ] && aws xray delete-group --group-arn "$GROUP_ARN" 2>/dev/null && echo "  Deleted group $GROUP_NAME"
+    rm -rf "$WORK_DIR"
+    echo "Cleanup complete."
+}
+
+# Step 1: Send a trace segment
+echo "Step 1: Sending trace segments"
+TRACE_ID=$(python3 -c "import time,random;print(f'1-{int(time.time()):08x}-{random.randbytes(12).hex()}')")
+NOW=$(python3 -c "import time;print(time.time())")
+END=$(python3 -c "import time;print(time.time()+0.5)")
+
+SEGMENT="{\"trace_id\":\"$TRACE_ID\",\"id\":\"$(cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 8 | head -n 1)\",\"name\":\"tutorial-service\",\"start_time\":$NOW,\"end_time\":$END,\"http\":{\"request\":{\"method\":\"GET\",\"url\":\"https://example.com/api/items\"},\"response\":{\"status\":200}}}"
+
+aws xray put-trace-segments --trace-segment-documents "$SEGMENT" \
+    --query 'UnprocessedTraceSegments' --output text
+echo "  Trace ID: $TRACE_ID"
+
+# Send a subsegment
+PARENT_ID=$(echo "$SEGMENT" | python3 -c "import sys,json;print(json.load(sys.stdin)['id'])")
+SUBSEG="{\"trace_id\":\"$TRACE_ID\",\"id\":\"$(cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 8 | head -n 1)\",\"name\":\"database-query\",\"start_time\":$NOW,\"end_time\":$END,\"parent_id\":\"$PARENT_ID\"}"
+aws xray put-trace-segments --trace-segment-documents "$SUBSEG" > /dev/null 2>&1
+echo "  Sent parent segment + subsegment"
+
+# Step 2: Get trace summaries
+echo "Step 2: Getting trace summaries"
+sleep 5
+START_TIME=$(python3 -c "import time;print(int(time.time()-60))")
+END_TIME=$(python3 -c "import time;print(int(time.time()))")
+aws xray get-trace-summaries \
+    --start-time "$START_TIME" --end-time "$END_TIME" \
+    --query 'TraceSummaries[:3].{TraceId:Id,Duration:Duration,Status:Http.HttpStatus,Method:Http.HttpMethod}' --output table 2>/dev/null || \
+    echo "  No traces found yet (indexing can take a few seconds)"
+
+# Step 3: Get full trace
+echo "Step 3: Getting full trace"
+aws xray batch-get-traces --trace-ids "$TRACE_ID" \
+    --query 'Traces[0].Segments[].{Id:Id}' --output table 2>/dev/null || \
+    echo "  Trace not yet indexed"
+
+# Step 4: Create a group
+echo "Step 4: Creating trace group: $GROUP_NAME"
+GROUP_ARN=$(aws xray create-group --group-name "$GROUP_NAME" \
+    --filter-expression 'service("tutorial-service")' \
+    --query 'Group.GroupARN' --output text)
+echo "  Group ARN: $GROUP_ARN"
+
+# Step 5: Get service graph
+echo "Step 5: Getting service graph"
+aws xray get-service-graph \
+    --start-time "$START_TIME" --end-time "$END_TIME" \
+    --query 'Services[].{Name:Name,Type:Type,Edges:Edges|length(@)}' --output table 2>/dev/null || \
+    echo "  No service graph available yet"
+
+echo ""
+echo "Tutorial complete."
+echo "Do you want to clean up all resources? (y/n): "
+read -r CHOICE
+if [[ "$CHOICE" =~ ^[Yy]$ ]]; then
+    cleanup
+else
+    echo "Manual cleanup:"
+    echo "  aws xray delete-group --group-arn $GROUP_ARN"
+fi

tuts/119-aws-glue-gs/README.md

@@ -0,0 +1,58 @@
+# Glue: Create a Data Catalog database and table
+
+Create an AWS Glue Data Catalog database, upload sample JSON data to S3, create an external table pointing to the data, and query the catalog.
+
+## Source
+
+https://docs.aws.amazon.com/glue/latest/dg/getting-started.html
+
+## Use case
+
+- **ID**: glue/getting-started
+- **Level**: beginner
+- **Core actions**: `glue:CreateDatabase`, `glue:CreateTable`, `glue:GetTable`, `glue:GetDatabases`, `glue:GetTables`
+
+## Steps
+
+1. Create a Glue database
+2. Create an S3 bucket and upload sample data
+3. Create an external table with JSON SerDe
+4. Describe the table
+5. List databases and tables
+
+## Resources created
+
+| Resource | Type |
+|----------|------|
+| `tut_db_<random>` | Glue database |
+| `tut_events` | Glue table |
+| `glue-tut-<random>-<account>` | S3 bucket (with sample data) |
+
+## Duration
+
+~12 seconds
+
+## Cost
+
+No charge. The Glue Data Catalog provides a free tier of 1 million objects stored and 1 million requests per month.
+
+## Related docs
+
+- [Getting started with AWS Glue](https://docs.aws.amazon.com/glue/latest/dg/getting-started.html)
+- [Defining databases in the Data Catalog](https://docs.aws.amazon.com/glue/latest/dg/define-database.html)
+- [Defining tables in the Data Catalog](https://docs.aws.amazon.com/glue/latest/dg/tables-described.html)
+- [AWS Glue pricing](https://aws.amazon.com/glue/pricing/)
+
+---
+
+## Appendix
+
+| Field | Value |
+|-------|-------|
+| Date | 2026-04-14 |
+| Script lines | 104 |
+| Exit code | 0 |
+| Runtime | 12s |
+| Steps | 5 |
+| Issues | None |
+| Version | v1 |

tuts/119-aws-glue-gs/REVISION-HISTORY.md

@@ -0,0 +1,8 @@
+# Revision History: 119-aws-glue-gs
+
+## Shell (CLI script)
+
+### 2026-04-14 v1 published
+- Type: functional
+- Initial version
+