Ethereum Transactions Crawler

Table of Contents

• Development Environment
• Project Structure
• Setup
• Start the application
• Running Tests
• API Rate Limit

Development Environment

• Developed on Ubuntu 22.04
• Python version 3.12.0

Project structure

ethereum_transactions_crawler/
│
├── app/
│   ├── __init__.py
│   ├── main.py
│   ├── run.py
│   └── templates/
│       ├── index.html
│       └── transaction_detail.html
│
├── tests/
│   ├── __init__.py
│   └── test_main.py
│
├── requirements.txt
├── .env
└── README.md

Setup

• Create a virtual environment in the root directory and activate it

  python -m venv venv
  source venv/bin/activate
  

• Install requirements

  pip install -r requirements.txt
  

• Create a .env file in root directory and in it declare the INFURA_URL environment variable

  INFURA_URL="https://mainnet.infura.io/v3/INFURA_PROJECT_ID"

Start the application

• Application is started from root directory with:

  python app/run.py
  

• You can access the application frontend here

Running tests

• You can run tests from root directory with:

  pytest
  

API Rate Limit

• Infura has a rate limit of 10 requests per second on their free plan. This is the biggest bottleneck in the program and can affect the performance of this application. This is especially obvious when retrieving a large amount of blocks.
• The RATE_LIMIT variable can be found in the app/main.py file and is by default set to 10. If one has a paid Infura account they should change this variable to improve the application performance.