Skip to content

Commit 333690a

Browse files
mwunderlmwunderl
committed
Add integration tutorials (batch 21)
1 parent 49f07d9 commit 333690a

9 files changed

Lines changed: 621 additions & 0 deletions

File tree

‎tuts/116-aws-xray-gs/README.md‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# X-Ray: Send traces and query them
2+
3+
Send trace segments to AWS X-Ray, query trace summaries, retrieve full traces, create a trace group, and view the service graph.
4+
5+
## Source
6+
7+
https://docs.aws.amazon.com/xray/latest/devguide/xray-api-sendingdata.html
8+
9+
## Use case
10+
11+
- **ID**: xray/getting-started
12+
- **Level**: intermediate
13+
- **Core actions**: `xray:PutTraceSegments`, `xray:GetTraceSummaries`, `xray:BatchGetTraces`, `xray:CreateGroup`
14+
15+
## Steps
16+
17+
1. Send a trace segment and subsegment
18+
2. Get trace summaries
19+
3. Get full trace details
20+
4. Create a trace group with a filter expression
21+
5. Get the service graph
22+
23+
## Resources created
24+
25+
| Resource | Type |
26+
|----------|------|
27+
| `tut-group-<random>` | X-Ray group |
28+
29+
## Duration
30+
31+
~10 seconds
32+
33+
## Cost
34+
35+
No charge. X-Ray provides a free tier of 100,000 traces recorded and 1,000,000 traces scanned per month.
36+
37+
## Related docs
38+
39+
- [Sending trace data to X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-api-sendingdata.html)
40+
- [Retrieving trace data](https://docs.aws.amazon.com/xray/latest/devguide/xray-api-gettingdata.html)
41+
- [Using groups in X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-console-groups.html)
42+
- [AWS X-Ray pricing](https://aws.amazon.com/xray/pricing/)
43+
44+
---
45+
46+
## Appendix
47+
48+
| Field | Value |
49+
|-------|-------|
50+
| Date | 2026-04-14 |
51+
| Script lines | 87 |
52+
| Exit code | 0 |
53+
| Runtime | 10s |
54+
| Steps | 5 |
55+
| Issues | Fixed parent_id extraction quoting |
56+
| Version | v1 |

‎tuts/116-aws-xray-gs/aws-xray-gs.md‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
# Send traces and query them with AWS X-Ray
2+
3+
## Overview
4+
5+
In this tutorial, you use the AWS CLI to send trace segments to AWS X-Ray, query trace summaries, retrieve full trace details, create a trace group with a filter expression, and view the service graph. You then delete the group during cleanup.
6+
7+
## Prerequisites
8+
9+
- AWS CLI installed and configured with appropriate permissions.
10+
- Python 3 installed (used to generate trace IDs and timestamps).
11+
- An IAM principal with permissions for `xray:PutTraceSegments`, `xray:GetTraceSummaries`, `xray:BatchGetTraces`, `xray:CreateGroup`, `xray:DeleteGroup`, and `xray:GetServiceGraph`.
12+
13+
## Step 1: Send a trace segment
14+
15+
Generate a trace ID and send a segment representing an HTTP request, followed by a subsegment representing a downstream call.
16+
17+
```bash
18+
TRACE_ID=$(python3 -c "import time,random;print(f'1-{int(time.time()):08x}-{random.randbytes(12).hex()}')")
19+
NOW=$(python3 -c "import time;print(time.time())")
20+
END=$(python3 -c "import time;print(time.time()+0.5)")
21+
22+
SEGMENT="{\"trace_id\":\"$TRACE_ID\",\"id\":\"$(openssl rand -hex 8)\",\"name\":\"tutorial-service\",\"start_time\":$NOW,\"end_time\":$END,\"http\":{\"request\":{\"method\":\"GET\",\"url\":\"https://example.com/api/items\"},\"response\":{\"status\":200}}}"
23+
24+
aws xray put-trace-segments --trace-segment-documents "$SEGMENT"
25+
```
26+
27+
X-Ray trace IDs follow the format `1-<unix epoch hex>-<96-bit random hex>`. Each segment needs a unique 64-bit hex ID, a name, and start/end times as Unix epoch floats.
28+
29+
Send a subsegment linked to the parent segment:
30+
31+
```bash
32+
PARENT_ID=$(echo "$SEGMENT" | python3 -c "import sys,json;print(json.load(sys.stdin)['id'])")
33+
SUBSEG="{\"trace_id\":\"$TRACE_ID\",\"id\":\"$(openssl rand -hex 8)\",\"name\":\"database-query\",\"start_time\":$NOW,\"end_time\":$END,\"parent_id\":\"$PARENT_ID\"}"
34+
aws xray put-trace-segments --trace-segment-documents "$SUBSEG"
35+
```
36+
37+
Subsegments use `parent_id` to link to their parent segment. This creates the hierarchy visible in the X-Ray trace map.
38+
39+
## Step 2: Get trace summaries
40+
41+
Query recent trace summaries. X-Ray takes a few seconds to index new traces.
42+
43+
```bash
44+
START_TIME=$(python3 -c "import time;print(int(time.time()-60))")
45+
END_TIME=$(python3 -c "import time;print(int(time.time()))")
46+
47+
aws xray get-trace-summaries \
48+
--start-time "$START_TIME" --end-time "$END_TIME" \
49+
--query 'TraceSummaries[:3].{TraceId:Id,Duration:Duration,Status:Http.HttpStatus,Method:Http.HttpMethod}' \
50+
--output table
51+
```
52+
53+
Trace summaries include the trace ID, duration, HTTP status, and response time. Use `--filter-expression` to narrow results by service name, status code, or annotation.
54+
55+
## Step 3: Get full trace details
56+
57+
Retrieve the complete trace including all segments and subsegments.
58+
59+
```bash
60+
aws xray batch-get-traces --trace-ids "$TRACE_ID" \
61+
--query 'Traces[0].Segments[].{Id:Id}' --output table
62+
```
63+
64+
`batch-get-traces` returns the raw segment documents. You can retrieve up to 5 trace IDs per call.
65+
66+
## Step 4: Create a trace group
67+
68+
Create a group that filters traces by service name.
69+
70+
```bash
71+
GROUP_NAME="tut-group-$(openssl rand -hex 4)"
72+
73+
GROUP_ARN=$(aws xray create-group --group-name "$GROUP_NAME" \
74+
--filter-expression 'service("tutorial-service")' \
75+
--query 'Group.GroupARN' --output text)
76+
echo "Group ARN: $GROUP_ARN"
77+
```
78+
79+
Groups let you organize traces by filter expression. Each group generates its own service graph and CloudWatch metrics.
80+
81+
## Step 5: Get the service graph
82+
83+
View the service graph for the time window containing your traces.
84+
85+
```bash
86+
aws xray get-service-graph \
87+
--start-time "$START_TIME" --end-time "$END_TIME" \
88+
--query 'Services[].{Name:Name,Type:Type,Edges:Edges|length(@)}' \
89+
--output table
90+
```
91+
92+
The service graph shows services as nodes and their connections as edges. It may take a few seconds after sending traces for the graph to populate.
93+
94+
## Cleanup
95+
96+
Delete the trace group. Trace data itself expires automatically based on your X-Ray retention settings (default 30 days).
97+
98+
```bash
99+
aws xray delete-group --group-arn "$GROUP_ARN"
100+
```
101+
102+
The script automates all steps including cleanup:
103+
104+
```bash
105+
bash aws-xray-gs.sh
106+
```
107+
108+
## Related resources
109+
110+
- [Sending trace data to X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-api-sendingdata.html)
111+
- [Retrieving trace data](https://docs.aws.amazon.com/xray/latest/devguide/xray-api-gettingdata.html)
112+
- [Using groups in X-Ray](https://docs.aws.amazon.com/xray/latest/devguide/xray-console-groups.html)
113+
- [AWS X-Ray pricing](https://aws.amazon.com/xray/pricing/)

‎tuts/116-aws-xray-gs/aws-xray-gs.sh‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
#!/bin/bash
2+
# Tutorial: Send traces and query them with AWS X-Ray
3+
# Source: https://docs.aws.amazon.com/xray/latest/devguide/xray-api-sendingdata.html
4+
5+
WORK_DIR=$(mktemp -d)
6+
LOG_FILE="$WORK_DIR/xray-$(date +%Y%m%d-%H%M%S).log"
7+
exec > >(tee -a "$LOG_FILE") 2>&1
8+
9+
REGION=${AWS_DEFAULT_REGION:-${AWS_REGION:-$(aws configure get region 2>/dev/null)}}
10+
if [ -z "$REGION" ]; then
11+
echo "ERROR: No AWS region configured. Set one with: export AWS_DEFAULT_REGION=us-east-1"
12+
exit 1
13+
fi
14+
export AWS_DEFAULT_REGION="$REGION"
15+
echo "Region: $REGION"
16+
17+
RANDOM_ID=$(openssl rand -hex 4)
18+
GROUP_NAME="tut-group-${RANDOM_ID}"
19+
20+
handle_error() { echo "ERROR on line $1"; trap - ERR; cleanup; exit 1; }
21+
trap 'handle_error $LINENO' ERR
22+
23+
cleanup() {
24+
echo ""
25+
echo "Cleaning up resources..."
26+
[ -n "$GROUP_ARN" ] && aws xray delete-group --group-arn "$GROUP_ARN" 2>/dev/null && echo " Deleted group $GROUP_NAME"
27+
rm -rf "$WORK_DIR"
28+
echo "Cleanup complete."
29+
}
30+
31+
# Step 1: Send a trace segment
32+
echo "Step 1: Sending trace segments"
33+
TRACE_ID=$(python3 -c "import time,random;print(f'1-{int(time.time()):08x}-{random.randbytes(12).hex()}')")
34+
NOW=$(python3 -c "import time;print(time.time())")
35+
END=$(python3 -c "import time;print(time.time()+0.5)")
36+
37+
SEGMENT="{\"trace_id\":\"$TRACE_ID\",\"id\":\"$(openssl rand -hex 8)\",\"name\":\"tutorial-service\",\"start_time\":$NOW,\"end_time\":$END,\"http\":{\"request\":{\"method\":\"GET\",\"url\":\"https://example.com/api/items\"},\"response\":{\"status\":200}}}"
38+
39+
aws xray put-trace-segments --trace-segment-documents "$SEGMENT" \
40+
--query 'UnprocessedTraceSegments' --output text
41+
echo " Trace ID: $TRACE_ID"
42+
43+
# Send a subsegment
44+
PARENT_ID=$(echo "$SEGMENT" | python3 -c "import sys,json;print(json.load(sys.stdin)['id'])")
45+
SUBSEG="{\"trace_id\":\"$TRACE_ID\",\"id\":\"$(openssl rand -hex 8)\",\"name\":\"database-query\",\"start_time\":$NOW,\"end_time\":$END,\"parent_id\":\"$PARENT_ID\"}"
46+
aws xray put-trace-segments --trace-segment-documents "$SUBSEG" > /dev/null 2>&1
47+
echo " Sent parent segment + subsegment"
48+
49+
# Step 2: Get trace summaries
50+
echo "Step 2: Getting trace summaries"
51+
sleep 5
52+
START_TIME=$(python3 -c "import time;print(int(time.time()-60))")
53+
END_TIME=$(python3 -c "import time;print(int(time.time()))")
54+
aws xray get-trace-summaries \
55+
--start-time "$START_TIME" --end-time "$END_TIME" \
56+
--query 'TraceSummaries[:3].{TraceId:Id,Duration:Duration,Status:Http.HttpStatus,Method:Http.HttpMethod}' --output table 2>/dev/null || \
57+
echo " No traces found yet (indexing can take a few seconds)"
58+
59+
# Step 3: Get full trace
60+
echo "Step 3: Getting full trace"
61+
aws xray batch-get-traces --trace-ids "$TRACE_ID" \
62+
--query 'Traces[0].Segments[].{Id:Id}' --output table 2>/dev/null || \
63+
echo " Trace not yet indexed"
64+
65+
# Step 4: Create a group
66+
echo "Step 4: Creating trace group: $GROUP_NAME"
67+
GROUP_ARN=$(aws xray create-group --group-name "$GROUP_NAME" \
68+
--filter-expression 'service("tutorial-service")' \
69+
--query 'Group.GroupARN' --output text)
70+
echo " Group ARN: $GROUP_ARN"
71+
72+
# Step 5: Get service graph
73+
echo "Step 5: Getting service graph"
74+
aws xray get-service-graph \
75+
--start-time "$START_TIME" --end-time "$END_TIME" \
76+
--query 'Services[].{Name:Name,Type:Type,Edges:Edges|length(@)}' --output table 2>/dev/null || \
77+
echo " No service graph available yet"
78+
79+
echo ""
80+
echo "Tutorial complete."
81+
echo "Do you want to clean up all resources? (y/n): "
82+
read -r CHOICE
83+
if [[ "$CHOICE" =~ ^[Yy]$ ]]; then
84+
cleanup
85+
else
86+
echo "Manual cleanup:"
87+
echo " aws xray delete-group --group-arn $GROUP_ARN"
88+
fi

‎tuts/119-aws-glue-gs/README.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# Glue: Create a Data Catalog database and table
2+
3+
Create an AWS Glue Data Catalog database, upload sample JSON data to S3, create an external table pointing to the data, and query the catalog.
4+
5+
## Source
6+
7+
https://docs.aws.amazon.com/glue/latest/dg/getting-started.html
8+
9+
## Use case
10+
11+
- **ID**: glue/getting-started
12+
- **Level**: beginner
13+
- **Core actions**: `glue:CreateDatabase`, `glue:CreateTable`, `glue:GetTable`, `glue:GetDatabases`, `glue:GetTables`
14+
15+
## Steps
16+
17+
1. Create a Glue database
18+
2. Create an S3 bucket and upload sample data
19+
3. Create an external table with JSON SerDe
20+
4. Describe the table
21+
5. List databases and tables
22+
23+
## Resources created
24+
25+
| Resource | Type |
26+
|----------|------|
27+
| `tut_db_<random>` | Glue database |
28+
| `tut_events` | Glue table |
29+
| `glue-tut-<random>-<account>` | S3 bucket (with sample data) |
30+
31+
## Duration
32+
33+
~12 seconds
34+
35+
## Cost
36+
37+
No charge. The Glue Data Catalog provides a free tier of 1 million objects stored and 1 million requests per month.
38+
39+
## Related docs
40+
41+
- [Getting started with AWS Glue](https://docs.aws.amazon.com/glue/latest/dg/getting-started.html)
42+
- [Defining databases in the Data Catalog](https://docs.aws.amazon.com/glue/latest/dg/define-database.html)
43+
- [Defining tables in the Data Catalog](https://docs.aws.amazon.com/glue/latest/dg/tables-described.html)
44+
- [AWS Glue pricing](https://aws.amazon.com/glue/pricing/)
45+
46+
---
47+
48+
## Appendix
49+
50+
| Field | Value |
51+
|-------|-------|
52+
| Date | 2026-04-14 |
53+
| Script lines | 104 |
54+
| Exit code | 0 |
55+
| Runtime | 12s |
56+
| Steps | 5 |
57+
| Issues | None |
58+
| Version | v1 |

0 commit comments

Comments
 (0)