Dada-Tech / nyc-orchestration-pipeline

Using Mage

NYC Pipeline Orchestration with Mage

Run

docker compose build

Finally, start the Docker container:

docker compose up

Now, navigate to http://localhost:6789 in your browser! Voila! You're ready to get started with the course.

What just happened?

We just initialized a new mage repository. It will be present in your project under the name magic-zoomcamp. If you changed the variable PROJECT_NAME in the .env file, it will be named whatever you set it to.

This repository should have the following structure:

.
├── mage_data
│   └── magic-zoomcamp
├── magic-zoomcamp
│   ├── __pycache__
│   ├── charts
│   ├── custom
│   ├── data_exporters
│   ├── data_loaders
│   ├── dbt
│   ├── extensions
│   ├── interactions
│   ├── pipelines
│   ├── scratchpads
│   ├── transformers
│   ├── utils
│   ├── __init__.py
│   ├── io_config.yaml
│   ├── metadata.yaml
│   └── requirements.txt
├── Dockerfile
├── README.md
├── dev.env
├── docker-compose.yml
└── requirements.txt

Assistance

1. Mage Docs: a good place to understand Mage functionality or concepts.
2. Mage Slack: a good place to ask questions or get help from the Mage team.
3. DTC Zoomcamp: a good place to get help from the community on course-specific inquiries.
4. Mage GitHub: a good place to open issues or feature requests.