[Course][Udemy][Gourav Shah] DevOps to MLOps Bootcamp: Build & Deploy ML Systems End-to-End [ENG, 2025]

Git
https://github.com/mlopsbootcamp/house-price-predictor

https://github.com/initcron/hprice-predictor
https://github.com/gouravshah/hprice-predictor/

Делаю:
2025.06.24

$ sudo apt install -y tree jq

03. Use Case and Environment Setup

06. Launching MLflow for Experiemnt Tracking

$ mkdir -p ~/projects/courses/mlops
$ cd ~/projects/courses/mlops/

$ git clone [email protected]:webmakaka/house-price-predictor.git
$ cd house-price-predictor/deployment/mlflow/
$ docker compose up

// OK!
http://localhost:5555

08. Setting up Python Virtual Environment with UV

$ curl -LsSf https://astral.sh/uv/install.sh | sh
$ uv venv --python python3.11
$ source .venv/bin/activate

$ cd ~/projects/courses/mlops/
$ uv pip install -r requirements.txt

09. Working with Jupyter Notebooks

$ code .

^P

ext install ms-python.python
ext install ms-toolsai.jupyter

В vscode

Run All -> notebooks\*.ipynb

В результате в MlFlow должны появиться эксперименты.

04. From Data to Models - Understanding Data Science with Feature Engineering

02. Learning Data Engineering

$ python src/data/run_processing.py \
  --input data/raw/house_data.csv \
  --output data/processed/cleaned_house_data.csv

06. Preparing for Model Experimentation

$ python src/features/engineer.py \
  --input data/processed/cleaned_house_data.csv \
  --output data/processed/featured_house_data.csv \
  --preprocessor models/trained/preprocessor.pkl

06. Packaging Model along with FastAPI Wrapper and Streamlit with Containers

03. Running Feature Engineering and Preprocessing Jobs

$ python src/features/engineer.py \
  --input data/processed/cleaned_house_data.csv \
  --output data/processed/featured_house_data.csv \
  --preprocessor models/trained/preprocessor.pkl

04. Building and Training Final Model with Configs from Data Scientists

$ python src/models/train_model.py \
  --config configs/model_config.yaml \
  --data data/processed/featured_house_data.csv \
  --models-dir models \
  --mlflow-tracking-uri http://localhost:5555

http://localhost:5555/#/models

В UI появилась зарегистрированная модель

$ tree

Появилась модель house_price_model.pkl

├── models
│   └── trained
│       ├── house_price_model.pkl
│       ├── preprocessor.pkl
│       └── README.md

06. Writing Dockerfile to package Model with FastAPI Wrapper

Код из репо выше

$ vi Dockerfile

FROM python:3.11-slim

WORKDIR /app

COPY src/api/ .

RUN pip install -r requirements.txt

COPY models/trained/*.pkl models/trained/

EXPOSE 8000

CMD [ "uvicorn",  "main:app",  "--host",  "0.0.0.0",  "--port",  "8000" ]

$ docker image build -t fastapi .

07. Debugging and Fixing Image Failures, Launch and Validate FastAPI

$ docker run --rm -it fastapi:latest bash

# uvicorn main:app --host 0.0.0.0 --port 8000
INFO:     Started server process [7]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

^C^D

$ docker run -idtP fastapi:latest

$ docker ps
CONTAINER ID   IMAGE                          COMMAND                  CREATED         STATUS         PORTS                                           NAMES
7b7be31c8f03   fastapi:latest                 "uvicorn main:app --…"   5 seconds ago   Up 4 seconds   0.0.0.0:32768->8000/tcp, [::]:32768->8000/tcp   silly_dirac

// OK!
// Открылся swagger
http://localhost:32768/docs

$ curl -X POST "http://localhost:32768/predict" \
-H "Content-Type: application/json" \
-d '{
  "sqft": 1500,
  "bedrooms": 3,
  "bathrooms": 2,
  "location": "suburban",
  "year_built": 2000,
  "condition": "fair"
}' | jq

response:

{
  "predicted_price": 487238.4,
  "confidence_interval": [438514.56, 535962.24],
  "features_importance": {},
  "prediction_time": "2025-06-24T10:36:53.604637"
}

08. Packaging and testing Streamlit App

$ cd streamlit_app

$ vi Dockerfile

FROM python:3.9-slim

WORKDIR /app

COPY  app.py requirements.txt .

RUN pip install -r requirements.txt

EXPOSE 8501

CMD  [ "streamlit", "run", "app.py", "--server.address=0.0.0.0" ]

$ docker image build -t webmakaka/streamlit:v1 .
$ docker login
$ docker push webmakaka/streamlit:v1

09. Packaging and Model Serving Infra with Docker Compose

$ vi docker-compose.yaml

services:
  fastapi:
    image: docker.io/webmakaka/fastapi:dev
    build:
      context: "./"
      dockerfile: "Dockerfile"
    ports:
      - "8000:8000"

  streamlit:
    image: docker.io/webmakaka/streamlit:dev
    build:
      context: "streamlit_app/"
      dockerfile: "Dockerfile"
    ports:
      - "8501:8501"
    environment:
      API_URL: "http://fastapi:8000"

$ docker compose build
$ docker compose up -d

// OK!
http://localhost:8501/

$ docker push docker.io/webmakaka/fastapi:dev
$ docker push docker.io/webmakaka/streamlit:dev

$ docker compose stop

07. Setting up MLOps CI Workflow with GitHub Actions

04. Writing an executung out first GitHub Actions Workflow

https://github.com/gouravshah/house-price-predictor3/tree/main/.github/workflows

hub.docker.com -> Login -> Account Settings -> Personal access tokens -> Generate new token

Description: github-webmakaka
Expiration date: none
Optional: Read & Write

Generate, Скопировать токен.

Github -> Repo -> house-price-predictor -> Settings -> Secrets and variables -> Actions -> Variables -> Repostitory variables -> New repository variable

DOCKERHUB_USERNAME

Secrets -> New repository secret

DOCKERHUB_TOKEN

$ vi .github/workflows/mlops-pipeline.yml

https://github.com/webmakaka/house-price-predictor/blob/main/.github/workflows/mlops-pipeline.yml

$ vi .github/workflows/streamlit-ci.yaml

https://github.com/webmakaka/house-price-predictor/blob/main/.github/workflows/streamlit-ci.yaml

08. Building Scalable Prod Inference Infrastructure with Kubernetes

04. Simplest way to build a 3 Node Kubernetes Cluster with KIND

Инсталляция и создание kubernetes кластера kind

06. Deploying Streamlit Frontent App with Kubernetes

$ kubectl create deployment streamlit --image=webmakaka/streamlit:latest --port=8501

07. Exposing the Streamlit App with Kubernetes NodePort Service

$ kubectl create service nodeport streamlit --tcp=8501 --node-port=30000

08. Creating Deployment Service for the Model wrapped in FastAPI

$ kubectl create deployment model --image=webmakaka/house-price-model:latest --port=8000

$ kubectl create service nodeport model --tcp=8000 --node-port=30100

// OK!
http://localhost:30100/docs
http://localhost:30000