software-engineering-playbook

🧠 Software Engineering Playbook

A complete methodology for building software projects: how to think and build from scratch to deployment, from CLI tools to full-stack apps.

License: MIT PRs Welcome

📌 About This Playbook

Most developers don’t get stuck because they lack tools. They get stuck because they lack a repeatable process. This playbook is that process, written down: a fixed sequence of steps that takes you from a blank folder to a deployed, tested, documented project.

Who it’s for

What’s inside

Stack at a glance

Path Stack Best for
Classic Python, Flask / Streamlit, SQLAlchemy, MySQL / PostgreSQL Fast portfolio projects, dashboards
Modern full-stack FastAPI, React, PostgreSQL, Alembic, Docker Production-style apps with a separate frontend
CLI Python stdlib, argparse, JSON Small tools, learning fundamentals

How to use it

  1. Read Part A once, since it changes how you approach projects.
  2. Before every project, walk through Part B in order.
  3. Use Part C as a reference while you work (Git, files, CI).
  4. Use Section 10 when you’re ready to ship.
  5. Copy the checklists in Part E into each new project.

📖 Table of Contents

PART A — The Philosophy

1. Why You Get Stuck: The Real Problem

You say: “I don’t know the process well enough yet.”

That’s the right starting point. Every junior developer feels the same. The difference is that most think the problem is:

Wrong. The real problem is:

You haven’t internalized the PROCESS. You know Python, Flask, FastAPI, SQL, React — but you don’t know the sequence of steps that takes you from blank to finished.

It’s like knowing how to use a hammer, nails, and a saw — but not knowing how to build a table. You know the tools, not the method.

What This Guide Will Change

Before:

After:

The difference: You think first, then you write.

2. The 3 Phases of Thinking

Every project goes through 3 phases of thinking:

🧠 Phase 1: Abstraction

“What do I want to do, in plain words?”

You think at a high level. You don’t touch code.

Example: “I want a CLI that tracks expenses and shows me totals.”

🔧 Phase 2: Decomposition

“What small pieces does it break into?”

You split the big thing into small pieces. Each piece becomes a module or a function.

Example:

Expense Tracker CLI:
├── Add expense
├── View expenses
├── Summary by category
└── Persistence (JSON)

💻 Phase 3: Implementation

“How do I turn this into code?”

You write actual code. The thinking is already done — now you execute.

Example: expense_tracker.py, storage.py, models.py

🔑 Rule: 80% thinking, 20% coding.

Juniors do the opposite: 5% thinking, 95% coding → they rewrite everything 3 times.

PART B — The Process (9 Steps)

3. Understanding & Decomposition

3.1 The 6 Questions Before Writing a Single Line

Write the answers in a NOTES.md before opening the editor.

# Question Example (Expense Tracker)
1 What problem am I solving? (1 sentence) “I want to track expenses and see totals.”
2 Who uses it and how? Me, via terminal
3 Inputs? amount: float, category: str, description: str
4 Outputs? Expense list, totals, messages
5 Constraints? amount > 0, category from a list, persistence
6 Edge cases? amount=”abc”, empty list, huge number

💡 For web apps, also answer: What are the API contracts (endpoints, request/response shapes)? Who is authenticated, and what can each user see? Which secrets does the app need?

3.2 Decomposition — The Split

Rule: If you can’t describe a sub-problem in 1-2 sentences, it’s too big.

Example:

"Track and analyze expenses"
├── 1. Add an expense
│   ├── 1a. Validate amount
│   ├── 1b. Validate category
│   └── 1c. Save
├── 2. View expenses
├── 3. Summary by category
└── 4. Persistence (JSON)

Each leaf → one function or one class.

3.3 When You Know Decomposition Is Done

If not → split more.

3.4 Decomposing a Full-Stack App

For a full-stack project, split first by layer, then by feature:

Full-Stack Expense Tracker
├── Backend (FastAPI + SQLAlchemy + Alembic)
│   ├── Auth (JWT / OAuth)
│   ├── Expenses CRUD & Caching
│   └── Database models & Migrations
└── Frontend (React + Tailwind + Vitest)
    ├── Authentication UI
    ├── Dashboard & Charts
    └── API Integration (Fetch/Axios)

4. Algorithmic Thinking

4.1 Pseudocode

Write the logic in plain English, using fixed keywords:

FUNCTION add_expense(amount, category):
    IF amount <= 0 THEN
        RETURN error
    
    IF category NOT IN valid_list THEN
        RETURN error
    
    CREATE new expense
    APPEND to list
    RETURN confirmation

Why: You think about the logic without worrying about Python syntax. When you write it in Python, the translation is nearly 1-to-1.

4.2 Dry Run — “Executing” on Paper

Take the pseudocode and “run” it by hand:

Input: amount = -5, category = "Food"
Step 1: -5 <= 0 → YES
Step 2: Returns error "Amount must be positive"
✅ Correct.

Rule: Dry run with 3 scenarios:

5. Design

5.1 UML Class Diagram

Shows:

┌─────────────────────┐
│      Expense        │
├─────────────────────┤
│ - amount: float     │
│ - category: str     │
│ - date: date        │
├─────────────────────┤
│ + to_dict() -> dict │
└─────────────────────┘

┌─────────────────────────────┐
│     ExpenseTracker          │
├─────────────────────────────┤
│ - expenses: list[Expense]   │
├─────────────────────────────┤
│ + add_expense(...)          │
│ + get_summary()             │
└─────────────────────────────┘

Questions:

  1. What “things” (nouns) exist? → Classes
  2. What data “belongs” to each thing?
  3. What actions (verbs) does it do?
  4. Do I need a class or a function? (If no state, a function is enough.)

5.2 Modular Architecture

Each file = one responsibility (SRP).

Classic layout (Flask / Streamlit / CLI):

project/
├── src/
│   ├── __init__.py
│   ├── main.py            # Entry point
│   ├── models/            # Domain models
│   ├── services/          # Business logic
│   ├── routes/            # HTTP endpoints
│   ├── storage/           # Persistence
│   └── ui/                # Presentation
├── tests/
├── requirements.txt
├── .env.example
├── .gitignore
└── README.md

Full-stack layout (FastAPI + React):

project/
├── backend/
│   ├── src/
│   │   ├── models/       # SQLAlchemy models
│   │   ├── schemas/      # Pydantic schemas
│   │   ├── services/     # Business logic
│   │   ├── routers/      # FastAPI endpoints
│   │   └── core/         # Config, database, security
│   ├── alembic/          # Database migrations
│   ├── tests/            # Pytest (unit & integration)
│   ├── Dockerfile
│   └── requirements.txt
├── frontend/
│   ├── src/              # React components & hooks
│   ├── tests/            # Vitest / E2E tests
│   └── package.json
├── docker-compose.yml
├── .env.example
├── .gitignore
└── README.md

Key Question: “If I change storage (e.g. JSON → DB), how many files do I touch?”

5.3 Data & API Design (Web Apps)

Before writing endpoints, write the contract:

Method Path Auth Request Response
POST /auth/register No {email, password} 201 {id, email}
POST /auth/login No {email, password} 200 {access_token}
GET /expenses JWT — 200 [Expense]
POST /expenses JWT {amount, category} 201 Expense

Also sketch an ERD (tables, primary keys, foreign keys) before creating any models.

6. Implementation

6.1 Rules for Functions

  1. One function, one job (SRP)
  2. Name = verb (add_expense, not expense_stuff)
  3. Type hints always
  4. Docstring on every non-obvious function
def add_expense(
    tracker: "ExpenseTracker",
    amount: float,
    category: str,
    description: str = "",
) -> "Expense":
    """Add a new expense after validating it.
    
    Args:
        tracker: The tracker to add to.
        amount: Amount > 0.
        category: Category from the allowed list.
        description: Optional.
    
    Returns:
        The new Expense.
    
    Raises:
        ValueError: If amount <= 0 or category is unknown.
    """
    if amount <= 0:
        raise ValueError("Amount must be positive")
    ...

6.2 Rules for Classes

  1. Represents a “thing” (not just a bag of functions)
  2. __init__ = initialization only (no heavy logic)
  3. Encapsulation — underscore for private attributes
  4. A class shouldn’t know too much about other classes

6.3 When to Use OOP vs Functions

Need a class if… Need a function if…
It has state (self.x) No state
You want inheritance No relation to others
You want many objects of the same type Pure logic

Example:

6.4 Configuration & Secrets

# core/config.py
import os
from dotenv import load_dotenv

load_dotenv()

class Settings:
    DATABASE_URL: str = os.getenv("DATABASE_URL", "sqlite:///./dev.db")
    SECRET_KEY: str = os.getenv("SECRET_KEY", "")
    DEBUG: bool = os.getenv("DEBUG", "false").lower() == "true"

settings = Settings()

6.5 Error Handling

6.6 Database Migrations

alembic revision --autogenerate -m "add expenses table"
alembic upgrade head      # apply
alembic downgrade -1      # roll back one step

6.7 Security Basics

7. Testing & Debugging

7.1 Testing — 3 Tests per Function

For every function, write at least 3 tests:

def test_add_expense_happy_path():
    """Normal scenario."""
    tracker = ExpenseTracker()
    tracker.add_expense(50.0, "Food")
    assert len(tracker.expenses) == 1

def test_add_expense_rejects_negative():
    """Edge case: negative."""
    with pytest.raises(ValueError):
        ExpenseTracker().add_expense(-10, "Food")

def test_add_expense_rejects_unknown_category():
    """Invalid input."""
    with pytest.raises(ValueError):
        ExpenseTracker().add_expense(10, "Unknown")

Why 3:

7.2 The Testing Pyramid

        /\
       /E2E\        few   — full user flows (slow)
      /------\
     /Integr. \     some  — API + database together
    /----------\
   /   Unit     \   many  — single functions (fast)
  /--------------\
Layer Tool (backend) Tool (frontend) What it checks
Unit pytest Vitest One function / component
Integration pytest + test DB (SQLite in-memory) React Testing Library Layers working together
E2E pytest + httpx / Playwright Playwright / Cypress Real user flows

Tips:

7.3 Debugging — 5 Steps

  1. Read the traceback — bottom to top
  2. Isolate — comment out pieces
  3. Print/Log intermediate values
  4. Form a hypothesis — “I think it’s X” → test it
  5. 20-25 minute rule — if stuck, ask for help

7.4 Logging vs Print

print() disappears. logging keeps history:

from loguru import logger

logger.info(f"Added expense: {amount}")
logger.warning(f"Invalid amount: {amount}")
logger.error(f"Failed to save: {e}")

Levels:

8. Refactoring

Refactoring = Change the STRUCTURE, not the BEHAVIOR.

⚠️ Do it AFTER the tests are green — tests are your safety net.

Questions:

  1. Code that repeats 2+ times? → Function
  2. Function > 20 lines? → Split it
  3. Magic numbers? → Constants
  4. Bad names? → Rename
  5. PEP 8? → black, ruff

8.1 Automate Quality with Pre-commit

Catch formatting and lint issues before they reach Git:

# .pre-commit-config.yaml
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    rev: v0.6.9
    hooks:
      - id: ruff
        args: [--fix]
      - id: ruff-format

pip install pre-commit
pre-commit install

9. Documentation

9.1 Docstrings vs Comments

Docstrings → the “what”:

def add_expense(...):
    """Adds a new expense."""

Comments → the “why”:

# BAD: states the obvious
i += 1  # increments i

# GOOD: explains why
i = 1  # skip header row

9.2 README Structure

# Project Name
Short description.

## Features
- ...

## Tech Stack
- ...

## Installation
```bash
git clone ...
pip install -r requirements.txt
```

## Usage
```bash
python main.py
```

## Screenshots
![...](...)

## Testing
```bash
pytest tests/ -v
```

## License
MIT

9.3 API Documentation

FastAPI generates interactive docs automatically. Link them in your README:

For Flask, document endpoints in the README as a table (method, path, auth, description).

10. Deployment

Deployment is not the last-minute step. It’s a skill with its own checklist. Follow the sections below in order.

10.1 Before Deploy — Checklist

10.2 Deployment Options

Project Type Where Time Free Tier
Streamlit Streamlit Community Cloud 5 min ✅
Flask Render / Railway 10 min ✅ (sleeps when idle)
FastAPI Render / Railway 10 min ✅ (sleeps when idle)
CLI GitHub only (or PyPI) — ✅
Frontend (React/Vite) Vercel / Netlify / GitHub Pages 5 min ✅
Database (PostgreSQL) Neon / Supabase 5 min ✅
Full Stack Vercel + Render + Neon 30 min ✅

⚠️ Free tiers change often. Check each provider’s current limits and pricing before you rely on them.

10.3 The Deployment Order for Full-Stack Apps

Always deploy from the bottom up, because each layer needs the one below it:

1. Database   (Neon / Supabase)   → get DATABASE_URL
        │
        ▼
2. Backend    (Render / Railway)  → get API URL
        │
        ▼
3. Frontend   (Vercel / Netlify)  → set VITE_API_URL to the API URL
        │
        ▼
4. Back to backend → set CORS_ORIGINS to the frontend URL

10.4 Step 1 — Deploy the Database (Neon or Supabase)

  1. Create an account at neon.tech or supabase.com.
  2. Create a new project and pick the region closest to your backend.
  3. Copy the connection string.
  4. Make sure it matches your driver and requires SSL:
postgresql+psycopg2://USER:PASSWORD@HOST/DBNAME?sslmode=require
  1. Save it as DATABASE_URL. You’ll paste it into your backend host in the next step.
  2. Run migrations against it once (from your machine, with the production URL exported temporarily):
export DATABASE_URL="postgresql+psycopg2://..."
alembic upgrade head

💡 Use connection pooling (Neon’s pooled connection string, or Supabase’s pooler) for serverless/free-tier setups to avoid exhausting connections.

10.5 Step 2 — Deploy the Backend (Render)

Prepare the repo:

@app.get("/health")
def health():
    return {"status": "ok"}

On Render:

  1. Go to render.com → New → Web Service → connect your GitHub repo.
  2. Configure:
Setting FastAPI Flask
Root Directory backend (if monorepo) repo root or subfolder
Runtime Python 3 Python 3
Build Command pip install -r requirements.txt pip install -r requirements.txt
Start Command uvicorn src.main:app --host 0.0.0.0 --port $PORT gunicorn "src:create_app()" --bind 0.0.0.0:$PORT
Health Check Path /health /health
  1. Under Environment, add your variables:
DATABASE_URL=postgresql+psycopg2://...?sslmode=require
SECRET_KEY=<long random string>
DEBUG=false
CORS_ORIGINS=http://localhost:5173
PYTHON_VERSION=3.12.0

Generate a strong secret locally:

python -c "import secrets; print(secrets.token_urlsafe(48))"
  1. Click Create Web Service. Wait for the build, then open https://your-service.onrender.com/docs (FastAPI) or /health.
  2. Copy the service URL. The frontend needs it.

⚠️ Free tier note: free web services sleep after a period of inactivity, so the first request can take 30-60 seconds. Mention this in your README so recruiters aren’t surprised.

Running migrations on deploy: if your plan supports a pre-deploy command, set it to alembic upgrade head. Otherwise, chain it into the start command:

alembic upgrade head && uvicorn src.main:app --host 0.0.0.0 --port $PORT

Railway alternative: New Project → Deploy from GitHub repo → add the same environment variables → set the start command. The steps are nearly identical.

10.6 Step 3 — Deploy the Frontend (Vercel)

  1. Go to vercel.com → Add New → Project → import your GitHub repo.
  2. Configure:
Setting Value
Root Directory frontend
Framework Preset Vite (or Create React App / Next.js)
Build Command npm run build
Output Directory dist (Vite) or build (CRA)
  1. Add the environment variable pointing at your backend:
VITE_API_URL=https://your-service.onrender.com
  1. Click Deploy. Vercel gives you a URL like https://your-app.vercel.app.

In the frontend, read the URL from the environment, never hardcode it:

const API_URL = import.meta.env.VITE_API_URL;
const res = await fetch(`${API_URL}/expenses`);

💡 Vite variables must start with VITE_ to be exposed to the browser. Never put secrets in them; everything in the frontend is public.

10.7 Step 4 — Connect Them (CORS)

The browser blocks requests from your frontend to your backend unless the backend allows that origin. Go back to Render and set:

CORS_ORIGINS=https://your-app.vercel.app

And in FastAPI:

import os
from fastapi.middleware.cors import CORSMiddleware

origins = os.getenv("CORS_ORIGINS", "").split(",")

app.add_middleware(
    CORSMiddleware,
    allow_origins=origins,
    allow_credentials=True,
    allow_methods=["*"],
    allow_headers=["*"],
)

Redeploy the backend. Never use allow_origins=["*"] together with credentials in production.

10.8 Deploying a Streamlit App

  1. Push the repo to GitHub. The entry file (e.g. app.py) and requirements.txt must be in the repo.
  2. Go to share.streamlit.io → New app.
  3. Select the repository, branch (main), and main file path (app.py).
  4. Open Advanced settings → Secrets and paste your secrets in TOML format:
DATABASE_URL = "postgresql://USER:PASSWORD@HOST/DBNAME?sslmode=require"
API_KEY = "your_api_key_here"
  1. Click Deploy. Your app goes live at https://your-app.streamlit.app.

Read secrets lazily (inside functions, not at import time) so tests and CI don’t fail with StreamlitSecretNotFoundError:

import streamlit as st

def get_database_url() -> str:
    return st.secrets["DATABASE_URL"]

Docker guarantees “it works on my machine” also means “it works everywhere.”

Backend Dockerfile:

FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["sh", "-c", "uvicorn src.main:app --host 0.0.0.0 --port ${PORT:-8000}"]

docker-compose.yml for local development:

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: dev
      POSTGRES_DB: mydb
    ports: ["5432:5432"]
    volumes:
      - pgdata:/var/lib/postgresql/data

  backend:
    build: ./backend
    ports: ["8000:8000"]
    env_file: .env
    depends_on: [db]

  frontend:
    build: ./frontend
    ports: ["5173:5173"]
    depends_on: [backend]

volumes:
  pgdata:

Verify locally before deploying:

docker compose up --build

Render and Railway can also deploy straight from your Dockerfile. Choose Docker as the runtime instead of Python.

10.10 After Deploy — Verification Checklist

10.11 Updating & Rolling Back

Update: merge to main → Render and Vercel redeploy automatically on push.

Rolling back:

10.12 Deployment Troubleshooting

Symptom Likely Cause Fix
Build fails: ModuleNotFoundError Dependency missing from requirements.txt Add it and push
App crashes: KeyError: DATABASE_URL Env var not set on the host Add it in the dashboard, redeploy
Address already in use / port error Not binding to $PORT Use --port $PORT --host 0.0.0.0
SSL connection required Missing sslmode=require Append it to DATABASE_URL
Frontend shows CORS error Backend doesn’t allow the frontend origin Set CORS_ORIGINS, redeploy backend
Frontend calls localhost in production Hardcoded API URL Use VITE_API_URL, redeploy frontend
relation "x" does not exist Migrations not applied Run alembic upgrade head on production
First request takes ~1 minute Free-tier cold start Expected; note it in the README
Works locally, fails on the host Python version mismatch Pin PYTHON_VERSION
Streamlit StreamlitSecretNotFoundError Secrets read at import time Read lazily; add secrets in app settings

11. Practice

The 3 Levels

Level 1 — Daily (15-20 min)

Level 2 — Weekly (small projects)

Level 3 — Core projects

Habits

  1. Re-read old code after 1 month
  2. Explain your code to someone (rubber duck)
  3. Read others’ code on GitHub
  4. Keep a postmortem — “what would I change?”
  5. Timed practice — 30 min/kata

🌱 The truth: Comfort comes around the 15th-20th project, not the 3rd. Don’t get discouraged.

PART C — The Technical Workflow

12. Git Workflow: Branches & Commits

12.1 The Branch Rule

Every feature = new branch. NEVER commit directly to main.

12.2 Branch Naming

Prefix When Example
chore/ Maintenance chore/initial-setup
feat/ New feature feat/storage-layer
fix/ Bug fix fix/config-ssl
test/ Tests test/unit-tests
docs/ Documentation docs/readme
refactor/ Refactor refactor/service-layer

12.3 The Flow (for every feature)

# 1. Sync
git checkout main
git pull

# 2. New branch
git checkout -b feat/feature-name

# 3. Work + commits (atomic!)
git add <file>
git commit -m "feat(scope): description"

# 4. Merge back
git checkout main
git merge --no-ff feat/feature-name -m "merge: description"
git branch -d feat/feature-name

# 5. Push
git push

💡 Team / PR workflow: instead of step 4, push the branch and open a Pull Request on GitHub. CI runs, you review, then merge via the GitHub UI.

12.4 Why --no-ff

Without: Flat history → you lose the structure. With: Creates a merge commit → branches are visible in git log --graph.

12.5 Commit Messages — Conventional Commits

Format:

<type>(<scope>): <short description>

Types:

Examples:

chore: add .gitignore for Python
feat(config): add env loader
feat(storage): add JSON file storage
test(storage): add unit tests
docs: rewrite README with screenshots
fix(config): correct indentation
merge: add storage layer

⛔ BAD:

"fixed stuff"
"wip"
"asdf"
"update"

✅ GOOD:

"feat(auth): add bcrypt password hashing"
"fix(api): handle 404 from GitHub API"
"test(tasks): add edge cases for empty input"

12.6 Atomic Commits — The Golden Rule

One commit = one logical change.

If you revert, you lose one thing — not five.

Example:

12.7 The Typical Branch Sequence for One Project

# Branch What You Do Commits
1 chore/initial-setup .gitignore, .env.example, README.md, folders 4-5
2 chore/requirements-and-config requirements.txt, config.py 2-3
3 feat/database-models SQLAlchemy models 2
4 feat/database-connection Engine + session 1-2
5 feat/services-layer Business logic 2-3
6 feat/routes-and-templates HTTP + UI 2-3
7 test/unit-tests Tests 2-3
8 chore/ci GitHub Actions 1
9 docs/final-readme README + screenshots 1-2
10 chore/add-license LICENSE 1

Total: ~20-25 commits, 10 branches.

12.8 When to Commit

Every time you:

Not:

12.9 Handy Git Commands

git status                      # what changed?
git log --oneline --graph       # visual history
git diff --staged               # review before committing
git restore --staged <file>     # unstage a file
git stash / git stash pop       # park work temporarily
git revert <commit>             # safely undo a commit (new commit)

13. The Essential Files of Every Project

13.1 .gitignore — First file!

Create it BEFORE anything else.

# Virtual Environment
.venv/
venv/
env/

# Secrets
.env
*.env

# Python cache
__pycache__/
*.pyc
*.pyo
.pytest_cache/
.ruff_cache/
.coverage
htmlcov/
*.egg-info/
build/
dist/

# Node (full-stack projects)
node_modules/

# OS junk
.DS_Store
Thumbs.db

# IDE
.vscode/
.idea/

# App data
data/
*.log
*.db
tasks.json

Why first: Prevents accidental commits.

⚠️ *.env also ignores .env.example-style names only if they end in .env. .env.example is not matched, so it stays tracked. Verify with git status.

13.2 .env.example — Template

What others need, WITHOUT the real secrets.

# Flask
SECRET_KEY=your_secret_key_here
FLASK_DEBUG=True

# Database
DB_HOST=localhost
DB_PORT=3306
DB_USER=root
DB_PASSWORD=your_password_here
DB_NAME=myapp
DATABASE_URL=mysql+pymysql://root:password@localhost:3306/myapp

# Web (full-stack)
CORS_ORIGINS=http://localhost:5173
VITE_API_URL=http://localhost:8000

Rule: Placeholders always. NEVER real credentials.

13.3 .env — Real Secrets

NOT pushed to Git (it’s in .gitignore).

SECRET_KEY=<generated, long and random>
DB_PASSWORD=<your real password>
DATABASE_URL=mysql+pymysql://root:<your real password>@localhost:3306/myapp

Creation: Copy-Item .env.example .env (Windows PowerShell) or cp .env.example .env (macOS/Linux) → fill in.

13.4 requirements.txt — Runtime

Flask>=3.0.0
SQLAlchemy>=2.0.0
python-dotenv>=1.0.0

13.5 requirements-dev.txt — Development

-r requirements.txt
pytest>=7.4.0
pytest-mock>=3.11.0
pytest-cov>=5.0.0
black>=24.0.0
ruff>=0.3.0

Why separate: Users who pip install don’t need pytest.

13.6 .venv/ — Virtual Environment

Creation:

python -m venv .venv
.venv\Scripts\Activate.ps1   # Windows
source .venv/bin/activate    # macOS/Linux

Why: Isolation — each project has its own packages.

⚠️ NEVER in Git (it’s in .gitignore).

13.7 README.md — The Showcase

Structure:

  1. Title + tagline
  2. Badges (Python, License, Tests, CI)
  3. Live Demo link
  4. Screenshots
  5. About (why you built it)
  6. Features (what it does)
  7. Tech Stack (what you use)
  8. Installation (how to run it)
  9. Usage (examples)
  10. Testing (how to run tests)
  11. Project Structure
  12. Deployment (where it runs, env vars needed)
  13. License

If you count 30 seconds: The recruiter reads the README before the code.

13.8 LICENSE — Legal

MIT License (most common for portfolios):

MIT License

Copyright (c) 2026 Your Name

Permission is hereby granted, free of charge, ...

13.9 pyproject.toml — Modern Packaging

[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"

[project]
name = "task-cli"
version = "0.1.0"
description = "A simple CLI task manager"
requires-python = ">=3.10"
license = { text = "MIT" }

[project.scripts]
task = "task_cli.main:main"

[tool.pytest.ini_options]
testpaths = ["tests"]

[tool.ruff]
line-length = 88

Why: pip install -e . → command task available everywhere.

13.10 Optional: Dockerfile + docker-compose.yml

See Section 10.9 for complete, ready-to-use files.

13.11 Optional: Makefile — Shortcuts

install:
	pip install -r requirements-dev.txt

test:
	pytest tests/ -v --cov=src

lint:
	ruff check . && black --check .

format:
	black . && ruff check --fix .

run:
	uvicorn src.main:app --reload

Now make test replaces a long command you’d otherwise forget.

14. CI/CD Pipeline

14.1 What CI Is

Continuous Integration = automatic tests on every push.

If they pass → ✅ If they fail → ❌ notification

Continuous Deployment (CD) = passing code is deployed automatically. Render and Vercel do this for you when connected to GitHub: push to main → auto-deploy.

14.2 GitHub Actions — tests.yml

Location: .github/workflows/tests.yml

name: Tests

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  test:
    name: Python $
    runs-on: ubuntu-latest
    
    strategy:
      fail-fast: false
      matrix:
        python-version: ["3.10", "3.11", "3.12"]
    
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python $
        uses: actions/setup-python@v5
        with:
          python-version: $
          cache: pip
      
      - name: Install dependencies
        run: |
          python -m pip install --upgrade pip
          pip install -r requirements.txt
          pip install pytest
      
      - name: Lint
        run: |
          pip install ruff
          ruff check .
      
      - name: Run tests
        run: pytest tests/ -v

14.3 Testing Against a Real Database in CI (Optional)

    services:
      postgres:
        image: postgres:16
        env:
          POSTGRES_PASSWORD: test
          POSTGRES_DB: testdb
        ports: ["5432:5432"]
        options: >-
          --health-cmd "pg_isready -U postgres"
          --health-interval 10s
          --health-timeout 5s
          --health-retries 5

    env:
      DATABASE_URL: postgresql+psycopg2://postgres:test@localhost:5432/testdb

Run alembic upgrade head before pytest to test migrations too.

14.4 Frontend CI (Vitest)

  frontend:
    runs-on: ubuntu-latest
    defaults:
      run:
        working-directory: frontend
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: npm
          cache-dependency-path: frontend/package-lock.json
      - run: npm ci
      - run: npm test -- --run
      - run: npm run build

14.5 Why Matrix 3 Versions

Shows it works on multiple Python versions → production-ready.

14.6 Badge in README

[![Tests](https://github.com/USER/REPO/actions/workflows/tests.yml/badge.svg)](https://github.com/USER/REPO/actions)

Will show:

14.7 When to Add CI

After:

Before:

14.8 Troubleshooting — Common Issues

Error Cause Fix
StreamlitSecretNotFoundError st.secrets at import Lazy init
ModuleNotFoundError: src Missing pyproject.toml Add config
pytest: not found Missing from requirements pip install pytest
KeyError: DATABASE_URL Tests want prod config TestConfig

Lesson: CI reveals design flaws before deploy.

15. Screenshots & Presentation

15.1 When to Take Screenshots

AFTER the project works live and has no bugs.

Before you:

15.2 How Many and Where

Project Type How Many What
Web app 3-4 Home + 2-3 features
CLI tool 2-3 --help + commands + output
API 1-2 Swagger UI + 1 request

15.3 How (Windows)

Method How
Win + Shift + S Snipping tool
Win + PrtScn Full screen → auto-save
Alt + PrtScn Active window only

15.4 Where to Store Them

project/
├── docs/
│   └── screenshots/
│       ├── home.png
│       ├── feature1.png
│       └── feature2.png

In README:

## 📸 Screenshots

| Home | Feature 1 |
|---|---|
| ![Home](docs/screenshots/home.png) | ![Feature](docs/screenshots/feature1.png) |

⚠️ On GitHub: Relative paths (docs/...), NOT absolute.

15.5 Best Practices

Put the live demo URL and one screenshot at the very top of the README, right under the title. Visitors decide in seconds whether to keep reading.

PART D — Case Studies

📊 Overview

# Project Stack Branches Commits Tests CI
1 Weather Dashboard Streamlit + API 10 ~25 13 ✅
2 To-Do App Streamlit + PostgreSQL 10 ~30 38 ✅
3 Flight Search Flask + MySQL 10 ~25 14 ✅
4 Task CLI Python CLI 7 ~20 33 ✅
5 Expense Tracker (full-stack) FastAPI + React + PostgreSQL 12+ ~35 pytest + Vitest ✅

🌤️ Weather Dashboard

Branches:

  1. chore/initial-setup — .gitignore, .env.example, README.md, folders
  2. chore/requirements-and-config — requirements.txt, config.py
  3. feat/weather-fetcher — API client
  4. feat/data-visualizer — Plotly charts
  5. feat/history-tracker — JSON persistence
  6. feat/streamlit-ui — UI
  7. test/unit-tests — pytest
  8. docs/final-readme — README + screenshots
  9. chore/add-license — MIT
  10. chore/ci — GitHub Actions

Skills: REST API, OOP, caching, testing, deployment

Deploy: Streamlit Community Cloud (Section 10.8)

✅ To-Do App

Branches:

  1. chore/initial-setup — setup
  2. chore/requirements-and-config — deps + config
  3. feat/database-models — User, Task
  4. feat/database-connection — Engine + session
  5. feat/auth-service — bcrypt + register/login
  6. feat/task-service — CRUD scoped per user
  7. feat/streamlit-auth-ui — Login/Register pages
  8. feat/streamlit-task-ui — Task list
  9. test/unit-tests — 38 tests
  10. docs/final-readme — README + screenshots
  11. chore/add-license — MIT
  12. chore/ci — GitHub Actions

Skills: Auth, PostgreSQL, per-user isolation, bcrypt, SQLAlchemy 2.0

Deploy: Streamlit Community Cloud + Neon / Supabase (Sections 10.4 and 10.8)

Branches:

  1. chore/initial-setup — setup
  2. chore/requirements-and-config — deps + config
  3. feat/database-models — 6 models + 2 associations
  4. feat/database-connection — lazy init engine
  5. feat/services-layer — business logic
  6. feat/routes-and-templates — Flask blueprints + Jinja2
  7. test/unit-tests — pytest + SQLite in-memory
  8. docs/final-readme — README + screenshots
  9. chore/add-license — MIT
  10. chore/ci — GitHub Actions

Skills: Complex SQL, multi-table JOINs, admin auth, Flask blueprints

Deploy: Render + managed database (Section 10.5)

📝 Task CLI

Branches:

  1. chore/initial-setup — setup
  2. chore/requirements-and-config — pytest only
  3. feat/storage — JSON persistence
  4. feat/task-logic — business logic
  5. feat/display — output formatting
  6. feat/cli — argparse entry point
  7. docs/final-readme — README + screenshots
  8. chore/ci — GitHub Actions

Skills: CLI, argparse, stdlib-only, atomic JSON writes

Deploy: GitHub only (optionally publish to PyPI)

💸 Expense Tracker (Full-Stack)

Stack: FastAPI, SQLAlchemy, Alembic, PostgreSQL, React, Tailwind, Docker, pytest, Vitest

Branches:

  1. chore/initial-setup — monorepo folders, .gitignore, .env.example
  2. chore/docker-compose — Postgres + backend + frontend containers
  3. feat/database-models — User, Expense
  4. chore/alembic-migrations — first migration
  5. feat/auth — JWT register/login, bcrypt
  6. feat/expenses-crud — endpoints scoped per user
  7. feat/caching — connection pooling + caching strategy
  8. feat/frontend-auth-ui — login/register pages
  9. feat/frontend-dashboard — charts + expense list
  10. test/backend-tests — pytest with DB fixtures
  11. test/frontend-tests — Vitest + Testing Library
  12. chore/ci — GitHub Actions (backend + frontend jobs)
  13. docs/final-readme — architecture, screenshots, live links
  14. chore/deploy — Neon + Render + Vercel

Skills: JWT/OAuth, migrations, Docker, full-stack integration, CORS, CI/CD, three-tier deployment

Deploy: Neon (DB) → Render (API) → Vercel (UI), exactly the order in Section 10.3

PART E — Checklists & Templates

16. Pre-Project Checklist

Before writing code:

17. Post-Project Checklist

Before deploy:

18. The Mental Model — Summary

       IDEA
        │
        ▼
  ┌──────────────────┐
  │ 1. UNDERSTANDING │  ← 30 min of thinking
  │    (6 questions) │
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 2. DECOMPOSITION │  ← Split into small pieces
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 3. PSEUDOCODE    │  ← Logic without syntax
  │    + DRY RUN     │
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 4. DESIGN        │  ← UML, ERD, API contracts
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 5. IMPLEMENT     │  ← Write code
  │  (branch-based)  │
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 6. TEST          │  ← 3 tests/function
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 7. REFACTOR      │  ← Clean up
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 8. DOCUMENT      │  ← README + docstrings
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 9. CI/CD         │  ← GitHub Actions
  └────────┬─────────┘
           │
  ┌────────▼─────────┐
  │ 10. DEPLOY       │  ← DB → API → UI → CORS
  └────────┬─────────┘
           │
           ▼
      PORTFOLIO

19. The Golden Rule

Think 80%, write 20%.

One commit = one change.

One branch = one feature.

Tests first, code after.

Never commit secrets.

Deploy from the bottom up: database, backend, frontend.

The README is 50% of the project.

20. The Most Important Thing

Comfort comes from repetition, not from reading.

You’re at #4. You’re not a beginner — you’re mid-way.

🤝 Contributing

Found a typo, a better tool, or a missing step? Open an issue or a pull request.

📄 License

MIT License — see LICENSE for details.

Built with ☕ + 🧠 — next time you start a project, read this file BEFORE touching the editor. 🚀