What the same task looks like across different tools

To make these distinctions concrete, let’s walk through the same scenario — “I have a repo on GitHub and I want Claude to add a utility function, write tests, and open a PR” — across Claude.ai, Claude Code, and GitHub Copilot.

# Install npm if needed (e.g., on a fresh WSL2 or Ubuntu setup)
sudo apt install npm

# Install and start Claude Code (sudo needed on Linux/WSL2)
sudo npm install -g @anthropic-ai/claude-code

# Install sandbox dependencies (WSL2/Linux only)
sudo apt-get update && sudo apt-get install bubblewrap socat

# Navigate to your project and launch Claude Code
# Note: in WSL2, your Windows files are at /mnt/c/Users/<username>/...
cd yourrepo && claude

Think in features, not projects

One of the biggest lessons from working with agentic coding tools: use them for feature-level development, not for building entire projects in one shot.

Why? Because agents work best with clear, well-scoped requests. The less clarity you provide, the more the agent has to guess — and guessing leads to:

• Agentic loops (trying approaches, failing, trying again)
• Drift from your intended architecture
• Wasted tokens and time
• Code that technically works but doesn’t match your vision

Precise requests get precise results. Instead of “build me a web app with auth,” try:

• “Add a login form component that submits to /api/auth/login and stores the JWT in a httpOnly cookie”
• “Write a pytest fixture that creates a test database with the schema from models.py”
• “Refactor the process_data function in pipeline.py to handle the case where input_df has missing columns”

Each of these is a single, well-defined task that an agent can execute without ambiguity.

Use CLAUDE.md as your control surface

CLAUDE.md is a markdown file you place in your project root that gives Claude Code persistent context about your project. Think of it as a README for the agent — it’s loaded automatically at the start of every session and shapes how Claude behaves. You can include things like:

• How your project is structured (key directories, entry points)
• Coding conventions (naming, formatting, patterns to follow or avoid)
• Testing and build commands
• Safety rules (“never force-push,” “don’t modify migrations/”)
• Links to docs or specs the agent should reference

Claude Code also supports CLAUDE.md files in subdirectories (loaded when Claude works in that directory) and a global ~/.claude/CLAUDE.md for preferences that apply across all projects. The file is advisory — Claude will follow these instructions in good faith, but they’re not enforced at the system level the way hooks or deny rules are. For anything safety-critical, back it up with a hook or deny rule.

This is one of the most underrated features — it’s your main lever for shaping how the agent behaves across sessions.

Prevent runaway loops:

## Testing requirements
- Always run the full test suite (`pytest tests/`) after making changes
- If tests fail, fix the failing tests before moving on
- Do not push code with failing tests

This single instruction saves enormous headaches. Without it, the agent might push broken code, you discover test failures in CI, and then you’re spending time fixing things that should have been caught locally.

Enforce project conventions:

## Code style
- Use type hints for all function signatures
- Follow the existing import ordering convention
- Do not add new dependencies without asking first

Limit destructive actions:

## Safety
- Never run `rm -rf` on any directory
- Never force-push to any branch
- Never modify files in the `config/production/` directory
- Always create a new branch for changes; never commit directly to main

Provide architectural context:

## Project structure
- API routes go in `src/routes/`
- Business logic goes in `src/services/`
- Database models are in `src/models/`
- Tests mirror the source structure under `tests/`

Good CLAUDE.md context reduces agentic loops — the agent spends less time exploring your project and more time doing useful work. But there’s a tradeoff: CLAUDE.md loads into every session, and a bloated file can hurt more than it helps.

See the official CLAUDE.md reference for the full spec, including file resolution order and advanced features.

Tune the permission dial

Claude Code’s permission system is the main thing that distinguishes it from tools that “just go.” By default, it asks before every file write, shell command, and git operation. This is safe but slow.

The key insight: permissions aren’t all-or-nothing. You can configure a spectrum:

• Start conservative — approve everything manually while you’re learning what the agent does
• Auto-approve low-risk actions — file reads, grep/search, test execution. These rarely cause harm and the prompts add friction without adding safety.
• Manually approve writes and git operations — this is where real damage can happen (overwriting files, force-pushing, committing secrets)
• Use CLAUDE.md safety rules as a second layer — even if you auto-approve shell commands, the agent will respect instructions like “never force-push”

The sweet spot for most developers: auto-approve reads and test runs, manually approve everything else. As you build trust with specific workflows, you can loosen further. See Anthropic’s permissions reference for the full rule syntax and available tool names.

Use branches and commit frequently

The non-negotiable: always work on a branch, never let an agent commit directly to main. Beyond that, there are two common workflows:

• Auto-commit freely, review at the PR stage. Let the agent commit (and even push) as it works. You review the full diff when you open the PR, just like you would with a human contributor. This keeps momentum high and works well when you have CI checks and a good test suite gating your merges.
• Commit manually after reviewing each change. Approve each commit yourself so you stay close to every change as it happens. This is safer when you’re learning the tool, working on sensitive code, or don’t yet have strong CI guardrails.

Either way, frequent commits help — they give you clean revert points if the agent goes off track. A good CLAUDE.md instruction like “commit after each completed task” keeps things granular regardless of which workflow you prefer.

Review everything (at the right level)

Agent-generated code isn’t exempt from review — but when you review is a matter of workflow. Some developers review each diff before committing; others let the agent run and review the full PR diff before merging. Both are valid. What matters is that someone (you, a teammate, or CI) checks the code before it lands on main:

• Read the diffs
• Check for security issues (hardcoded secrets, SQL injection, etc.)
• Verify it matches your architectural patterns
• Make sure it doesn’t introduce unnecessary complexity

Give the agent a way to verify its own work

This is the single highest-leverage thing you can do. Claude performs dramatically better when it can check its own output — running tests, comparing screenshots, validating behavior — rather than relying on you as the only feedback loop.

• Include test cases in your prompt: “Write a validateEmail function. Test cases: user@example.com → true, invalid → false, user@.com → false. Run the tests after implementing.”
• Ask it to verify UI changes visually: “[paste screenshot] Implement this design. Take a screenshot of the result and compare it to the original.”
• Point to the symptom, not just the fix: “The build fails with this error: [paste error]. Fix it and verify the build succeeds. Address the root cause, don’t suppress the error.”

The more you invest in making your verification rock-solid (a good test suite, a linter, a build check), the more autonomously the agent can work.

Explore first, then plan, then code

For complex tasks, resist the urge to let Claude jump straight to implementation. Use Plan Mode (toggle with Shift+Tab) to separate exploration from execution:

1. Explore: In Plan Mode, Claude reads files and answers questions without making changes. “Read src/auth/ and understand how we handle sessions and login.”
2. Plan: Ask Claude to create an implementation plan. “I want to add Google OAuth. What files need to change? Create a plan.”
3. Implement: Switch back to Normal Mode and let Claude execute the plan, verifying against tests.
4. Commit: Ask Claude to commit with a descriptive message.

Skip this for small, clear tasks — if you could describe the diff in one sentence, just ask Claude to do it directly. Planning is most useful when you’re uncertain about the approach or the change touches multiple files.

Manage context aggressively

Claude’s context window is your most important resource. As it fills up with conversation history, file contents, and command outputs, performance degrades — Claude may “forget” earlier instructions or make more mistakes. (This section is adapted from Anthropic’s official best practices.)

• Use /clear between unrelated tasks — a clean context dramatically improves quality
• Use /compact to summarize long conversations — run /compact focus on the API changes to keep what matters and discard the rest
• Delegate exploration to subagents — when Claude needs to read dozens of files to investigate something, have it use a subagent. The subagent works in its own context and returns a summary, keeping your main conversation lean.
• Run /context to see what’s consuming your context window (MCP servers can be surprisingly expensive)
• Course-correct early — if Claude is going in the wrong direction, interrupt with Esc rather than letting it generate more output that clutters context. After two failed corrections, /clear and start fresh with a better prompt.

Extend Claude Code with skills, hooks, and MCP

Beyond CLAUDE.md, Claude Code has a rich extension system for customizing behavior:

• Skills — reusable knowledge and workflows. Create a /deploy skill that runs your deployment checklist, or an API conventions skill that Claude loads when working on your endpoints. Skills load on demand, so they don’t bloat every session like CLAUDE.md does.
• Hooks — deterministic scripts that run at specific points in Claude’s workflow. Unlike CLAUDE.md instructions (which are advisory), hooks are guaranteed to fire. Use them for things like running ESLint after every file edit or blocking writes to a migrations/ directory.
• MCP — connect Claude to external services. Query your database, post to Slack, control a browser, or pull issues from your project tracker — all from within a Claude Code session.
• Subagents — isolated workers with their own context. Useful for research tasks, code review, or any work where you don’t want the intermediate steps cluttering your main conversation.

Start with CLAUDE.md for your core conventions. Add skills when you find yourself repeating the same workflows. Add hooks when you need guaranteed automation. Add MCP when you need external integrations. For a deeper dive, see Anthropic’s extension system overview, which covers when to use each mechanism.

What does this actually cost?

There are two ways to pay for Claude Code: subscription plans (fixed monthly cost) or API tokens (pay-per-use). Most individuals should start with a subscription; API pricing is better for automation and CI/CD pipelines. (Pricing details adapted from Anthropic’s pricing page and Claude Code cost management docs — verify current prices, as they change frequently.)

Subscription plans (as of early 2026):

With subscription plans, you never get a surprise bill — you hit rate limits instead. The /cost command shows your token usage in a session, but on a subscription plan this is informational only; it doesn’t affect your bill.

API token pricing (pay-per-use):

Note that output tokens cost 5× more than input tokens across all models — and code generation is output-heavy. Also, requests exceeding 200K input tokens are charged at 2× input / 1.5× output rates, which matters for large codebases. Prompt caching can reduce input costs by up to 90% on repeated system prompts, and the Batch API offers a 50% discount for async processing.

Anthropic reports that the average Claude Code user on API pricing spends roughly $6/day, with 90% of users under $12/day. That translates to $100–200/month for active development with Sonnet. But averages hide a lot of variance — one developer documented a single session that hit ~$150/hour running multiple parallel agents. One detailed usage report showed ~892K output tokens vs ~45K input tokens in a single month on a mix of Opus and Sonnet, costing ~$1,248.

Rule of thumb: If your monthly API costs would exceed $60–80, Max 5x is cheaper. If they’d exceed $150, Max 20x is the clear winner.

For comparison, GitHub Copilot runs $10–$39/month depending on tier, with usage-based pricing for premium models beyond included allowances.

Watch out for runaway costs

Agentic workflows can burn through tokens fast, especially when things go wrong:

• Agentic loops: A vague prompt can send the agent into cycles of trying approaches, failing, reading more files, and trying again — each loop consuming thousands of tokens.
• Context accumulation: As your conversation grows, every new message includes the full context window — so the 50th message in a session costs far more than the 1st. Use /clear between unrelated tasks and /compact to summarize long conversations.
• Parallel sessions: Running multiple Claude Code sessions simultaneously (especially on the web or with agent teams) multiplies your token consumption proportionally. Five parallel sessions = 5× the cost.
• Extended thinking: Thinking tokens are billed as output tokens. A complex Opus session with deep reasoning can generate thousands of thinking tokens per turn.

How to protect yourself:

• On a subscription: You can’t overspend, but you can hit rate limits mid-task. Monitor with /cost and plan your usage around your limit.
• On API pricing: Set spending alerts and hard limits on your Anthropic account. Use separate API keys for different projects so you can track spending.
• In both cases: Use /cost to monitor token usage mid-session. If a session is getting expensive, /clear and start fresh with a more specific prompt. Break large tasks into focused sessions.

For UW-Madison researchers: institutional cloud benefits

If you’re at UW-Madison (or a similar research institution), routing AI API costs through a UW-provisioned cloud account offers two main benefits: institutional billing (charges go to your cloud project, not your personal card — important for grants and shared budgets) and lower overhead on grants (UW’s Cloud Computing Pilot cuts F&A from 55.5% to 26%, saving ~$2,950 per $10,000 in cloud spending). NIH-funded researchers may get additional discounts through STRIDES. Note that these savings are on the overhead and billing side — Anthropic’s per-token pricing is the same whether you route through Vertex AI, Bedrock, or the direct API. Also note that institutional cloud agreements cover the cloud provider’s services — they do not extend to Anthropic’s data handling (see Data Privacy below).

Contact your department’s IT staff or Research Computing to ask about available cloud credits and whether AI API costs are eligible.

Strategies to keep costs down

• Be specific in your prompts — vague requests lead to more agentic loops, which means more tokens. “Add a login form” costs more than “Add a React component at src/components/LoginForm.tsx that posts email/password to /api/auth/login”
• Use /clear aggressively — reset context between unrelated tasks. A clean context means fewer input tokens per message.
• Use /compact — summarize long conversations to free up context space without losing key information
• Use the right model for the task — Haiku or Sonnet for straightforward tasks, reserve Opus for complex reasoning
• Break large tasks into smaller sessions — each focused session is cheaper than one sprawling conversation that loses context and re-reads files
• Use CLAUDE.md to provide project context upfront — this reduces the amount of exploration the agent needs to do
• Delegate exploration to subagents — they run in isolated context and return summaries, keeping your main session lean
• Monitor session costs — run /cost periodically to see where you stand

For more detail, see Anthropic’s cost management guide.

Energy and environmental considerations

Agentic coding is more compute-intensive than a simple chat query or web search. A single LLM text query now uses roughly 0.3 Wh — about the same as a Google search — thanks to hardware improvements and model optimization. But an agentic coding session chains hundreds or thousands of such calls together as the agent reads files, reasons, writes code, runs commands, and iterates.

How much energy does agentic coding actually use?

A detailed analysis by Simon P. Couch estimated Claude Code’s energy footprint at roughly 41 Wh per session — over 130× a single chat query. A heavy day of usage (multiple sessions, parallel agents) can reach ~1,300 Wh/day. To put that in perspective:

So a heavy day of agentic coding is roughly equivalent to running your dishwasher — modest at the individual level, but significant in aggregate.

The bigger picture:

• The IEA projects that global data center electricity consumption will roughly double from ~415 TWh in 2024 to over 945 TWh by 2030, driven largely by AI workloads. In the US, data centers are projected to consume more electricity than all energy-intensive manufacturing combined (aluminum, steel, cement, chemicals) by 2030.
• An estimated 60–90% of AI computing energy goes to inference (running models), not training. Training grabs headlines, but inference — every agentic session, every chat query — is where the ongoing energy cost lives.
• Cloud providers are investing in renewable energy, but coverage varies. Anthropic has pledged to offset energy costs and invested in grid optimization research, though the company lacks formal carbon reduction targets and a significant portion of new capacity is natural gas powered.
• On the efficiency side, a University of Rhode Island study found Claude Sonnet to be among the most energy-efficient frontier models, and energy per token has improved ~120× from GPT-3 to current models due to hardware and architecture advances.

What this means for you:

This doesn’t mean you shouldn’t use agentic tools — the productivity gains can be substantial, and the energy per unit of useful output may be better than the alternative (a human developer running builds, searching docs, and context-switching for hours). But it’s a reason to be intentional: don’t let an agent spin in wasteful loops when a well-scoped prompt would get the job done in one pass. Efficient prompting is both cheaper and greener.

Is your data used for model training?

Important nuances for consumer plans:

• Safety exception: Even if you disable training, conversations flagged for safety review may still be used to improve Anthropic’s ability to detect and enforce their Usage Policy (e.g., training safeguard models).
• What’s included: When training is enabled, Anthropic may use the entire conversation — prompts, outputs, custom styles, and conversation preferences.
• What’s excluded: Raw content from connectors (Google Drive, MCP servers) is not included in training data, unless you directly copy that content into your conversation.
• Feedback (thumbs up/down): Submitting feedback stores the full related conversation for up to 5 years, de-linked from your user ID. This data may be used for training regardless of your training setting.

For researchers with sensitive or restricted data: Routing through a cloud provider (Vertex AI, Bedrock) ensures your data is not used for training and limits retention to 30 days — but your prompts still reach Anthropic’s infrastructure for inference. UW-Madison has agreements with Google, AWS, and Microsoft for their cloud services, but does not yet have a direct data-use agreement with Anthropic. This means cloud routing alone does not provide UW-sanctioned data protections for restricted data (HIPAA/PHI, FERPA, CUI, export-controlled, or data under a DUA that prohibits third-party processing). Avoid using Claude Code with restricted data until a formal UW-Anthropic agreement is in place. For general, non-sensitive research code, cloud-routed Claude Code is fine to use today. UW is actively exploring institutional Anthropic licenses and data agreements. Enterprise customers can negotiate zero-data retention (ZDR) agreements where Anthropic stores nothing after the API response. See our Cloud Setup Guide for how UW-Madison researchers can use institutional cloud accounts (GCP or AWS) and for more details on data sensitivity considerations.

Can Anthropic employees see your code?

Not by default. Employee access to conversation data requires one of:

• You submit feedback (thumbs up/down, /bug command) — the full related conversation becomes reviewable, stored for up to 5 years (de-linked from your user ID for thumbs up/down)
• A trust & safety investigation — if Anthropic’s automated systems flag a policy violation (this data may also be used for training safeguard models)
• Explicit consent — you voluntarily share data with Anthropic

Under commercial terms (API, Vertex AI, Bedrock), access is further restricted by contractual obligations.

What about the web version?

When you use Claude Code on the web (claude.ai/code), your GitHub repo is cloned into an ephemeral VM. The VM is destroyed when the task completes — there’s no persistent repo storage between sessions. The same retention policies above apply to any code Claude reads during the session.

Telemetry and error reporting

Claude Code sends operational telemetry (latency, reliability metrics — no code or file paths) to Statsig, and error reports to Sentry. These are enabled by default on the direct Claude API but disabled by default on Vertex AI, Bedrock, and Foundry.

To opt out individually: DISABLE_TELEMETRY=1, DISABLE_ERROR_REPORTING=1, DISABLE_BUG_COMMAND=1, or CLAUDE_CODE_DISABLE_FEEDBACK_SURVEY=1. To disable all non-essential traffic at once: CLAUDE_CODE_DISABLE_NONESSENTIAL_TRAFFIC=1. These can be set in your settings.json.

For a detailed breakdown by provider, see the Cloud Setup Guide — Data Usage & Privacy.

Further reading on data privacy

• Is my data used for model training? — Anthropic Privacy Center
• How long do you store my data? — retention periods by account type
• Data usage — Claude Code docs — what Claude Code specifically transmits and how cloud sessions handle your repo
• Security — Claude Code docs — prompt injection safeguards, data retention, and web session isolation
• How do I change my model improvement privacy settings? — step-by-step opt-out instructions
• How does Anthropic protect personal data? — security practices and encryption

Use permissions and deny rules

Claude Code has a built-in permissions system that controls what it can do. In the default mode, it asks for approval before file writes, shell commands, and git operations. You can customize this with rules in settings.json:

• deny — hard block. Claude can’t use the tool, period. Deny rules always win, even if you accidentally click “always allow” on a prompt.
• allow — auto-approve. Skips the approval prompt for things you trust (e.g., git add, pytest).

Deny rules are your most important security layer. They protect sensitive paths — SSH keys, cloud credentials, .env files — regardless of what the agent tries to do. The approval prompt is your first line of defense; deny rules are the backup that can’t be bypassed.

{
  "permissions": {
    "deny": [
      "Read(//home/youruser/.ssh/**)",
      "Edit(//home/youruser/.ssh/**)",
      "Read(//home/youruser/.aws/**)",
      "Edit(//home/youruser/.aws/**)",
      "Read(./.env)",
      "Edit(./.env)",
      "Bash(rm -rf *)",
      "Bash(curl:*)",
      "Bash(wget:*)",
      "Bash(cat:*)"
    ]
  }
}

See the Cloud Setup Guide’s security section for a full walkthrough with platform-specific examples, or the official permissions docs for the complete rule syntax.

Enable Claude Code’s built-in sandbox

Claude Code’s built-in sandbox uses OS-level isolation (Linux namespaces / macOS Seatbelt) to restrict what shell commands can do — limiting filesystem writes to your project directory and blocking unauthorized network requests. This is separate from running inside a container (covered below). It’s lightweight, adds negligible overhead, and Anthropic’s internal testing found it reduced permission prompts by 84% while increasing security. Use it alongside deny rules for the strongest protection — Anthropic calls this “defense in depth”.

Scope your credentials

Even with deny rules and sandboxing, it’s good practice to limit what credentials the agent has access to in the first place.

Use minimal-scope tokens. Create fine-grained GitHub tokens scoped to only the repos and permissions the agent needs. If it only pushes to one repo, don’t give it access to your entire account. Use a bot account for agent-driven git operations, and generate dedicated deploy keys rather than reusing your personal SSH keys.

Set spending limits on API keys and use separate keys from your personal or production ones.

Add secrets to .gitignore — .env, credentials.json, *.pem, *.key, .netrc — before the agent ever runs. Once a secret is committed, it’s in the history. (But note: .gitignore prevents committing secrets, not reading them. Deny rules are what actually block the agent from accessing sensitive files.)

Consider containers for CI/CD and headless environments

For interactive development, the built-in sandbox is the right choice — it’s what Anthropic recommends, it’s lightweight, and combined with deny rules it provides strong isolation without any setup overhead. You don’t need Docker for local coding sessions.

Containers solve a different problem: unattended, non-interactive execution where there’s no human to approve permission prompts. In CI/CD pipelines, GitHub Actions, and headless automation, running Claude Code inside a container lets you use --dangerously-skip-permissions safely — the container itself is the isolation boundary, so there’s nothing outside it to damage. This is also the pattern Anthropic’s own GitHub Actions and GitLab CI/CD integrations use.

Options include a plain Docker container with your project mounted as a volume, Docker sandboxes (microVM-based isolation), or cloud sandbox platforms like E2B. For CI pipelines, ephemeral containers that are destroyed after each run are the safest option — nothing persists between runs.

Don’t stack them. The built-in sandbox and Docker containers are alternative isolation strategies. Running bubblewrap inside Docker introduces nested sandbox complexity without meaningful security benefit. Pick one: sandbox for interactive work, containers for headless automation.

Watch for prompt injection and runaway agents

Prompt injection is when an agent reads a file or message that contains hidden instructions designed to hijack its behavior. A malicious README.md, issue body, or .docx attachment could trick the agent into exfiltrating files or running harmful commands. Be especially cautious when pointing an agent at untrusted repositories or external content. Deny rules and sandboxing are your main defenses here — they limit what the agent can do even if it’s been tricked.

Runaway agents burn tokens and make unwanted changes when they get stuck in loops. Commit your work frequently so you can recover from mistakes, set spending limits on your API keys, and don’t hesitate to interrupt (Ctrl+C) and redirect. Set up git hooks or CI checks as safety nets — for example, preventing force-pushes to main.

Never give an agent unsupervised access to production systems, databases, or deployment pipelines.

Running Claude Code remotely

You don’t have to run Claude Code on your local machine. Running it over SSH on a cloud VM or remote server keeps your local system untouched and gives you access to more powerful hardware. For CI/CD integration — running Claude Code in GitHub Actions, GitLab CI, or similar systems — see the container discussion in Security fundamentals above, plus the official docs for GitHub Actions and GitLab CI/CD.

A note for GitLab users

Many teams — including many at UW-Madison — use GitLab rather than GitHub. Claude Code works with GitLab, but the integration is less mature than the GitHub experience.

What works well:

• Claude Code CLI with GitLab repos — the core experience (reading code, editing files, running commands) works identically regardless of your git host. Claude Code operates on your local checkout, so the remote platform doesn’t matter for day-to-day coding.
• GitLab CI/CD integration — Anthropic provides official documentation for running Claude Code in GitLab CI/CD pipelines, including merge request review and test scaffolding.
• Git operations — push, pull, branching, and committing all work normally since these are standard git operations.

What’s different or limited compared to GitHub:

• No native GitLab integration in Claude Code’s Slack bot — the Slack integration currently only supports GitHub repos. GitLab support is an open feature request.
• No @claude mention in GitLab issues/MRs — GitHub Copilot’s coding agent lets you assign issues to Copilot or mention it in PRs. There’s no equivalent native integration for GitLab yet, though GitLab is working on it.
• Community-built CI/CD tooling — while official docs exist, you may find yourself using community solutions to replicate the smoother GitHub Actions experience.
• Self-hosted GitLab — if your institution runs a self-hosted GitLab instance, be aware that Claude Code sends code context to Anthropic’s API for processing. This may raise compliance concerns depending on your institution’s data policies.

Practical advice: The CLI workflow is essentially identical — focus your setup effort on CI/CD integration. For MR review automation, use Claude Code in your .gitlab-ci.yml with the claude -p (prompt) flag for non-interactive pipeline usage. If your institution has data sensitivity requirements, check with your IT governance team before sending code to external APIs — this applies to all cloud-based AI coding tools, not just Claude.

Official documentation and guides

• Best Practices for Claude Code — Anthropic’s official guide, covering context management, prompt patterns, CLAUDE.md, and scaling across parallel sessions
• How Claude Code Works — the agentic loop architecture, built-in tools, and how Claude interacts with your project
• Extend Claude Code — when to use CLAUDE.md vs skills vs subagents vs hooks vs MCP
• Common Workflows — step-by-step guides for debugging, refactoring, testing, creating PRs, and more
• Claude Code on the Web — running Claude Code tasks asynchronously on cloud infrastructure
• Claude Code Desktop — the desktop GUI with visual diffs, parallel sessions, and managed updates
• Claude Code Sandboxing Documentation — reference for configuring Claude Code’s built-in sandboxing, including OS-level primitives (Linux bubblewrap, macOS Seatbelt) and deny rules for sensitive files
• Making Claude Code More Secure and Autonomous — Anthropic Engineering’s deep-dive into their dual-layer sandboxing architecture (filesystem + network isolation)
• Mitigating the Risk of Prompt Injections — Anthropic Research on defending AI agents against prompt injection, including their use of reinforcement learning to build injection robustness into Claude
• GitHub Copilot: Meet the New Coding Agent — GitHub’s announcement of their enterprise-ready coding agent that spins up secure environments via GitHub Actions
• GitHub Copilot Coding Agent 101 — GitHub’s getting-started guide for agentic workflows, including environment setup and PR creation
• What’s New with GitHub Copilot Coding Agent — latest updates including self-review, security scanning, and custom agents

Community voices and analysis

• Agentic Engineering Patterns — Simon Willison’s guide to coding practices for getting the best results from agents like Claude Code and Codex. He frames this as “expertise amplification, not expertise replacement”
• A Guide to Which AI to Use in the Agentic Era — Ethan Mollick’s updated guide arguing that “using AI” now means agents with tools, not chatbots, and that users must think in terms of Models, Apps, and Harnesses
• How I Use Claude Code (+ My Best Tips) — practical walkthrough from Builder.io on real-world Claude Code workflows
• The Complete Guide to Agentic Coding in 2026 — broad overview comparing tools, workflows, and team strategies
• Redefining the Software Engineering Profession for AI — ACM opinion piece on how AI amplifies senior talent but risks leaving junior developers without the chance to develop architectural intuition

Tool comparisons

• Cursor vs Windsurf vs Claude Code in 2026 — hands-on comparison arguing Cursor has the best IDE UX, Claude Code leads on deep reasoning and terminal-first workflows, and Windsurf offers the best value

Benchmarks and leaderboards

Agentic coding benchmarks are evolving rapidly. These track how well different models and agent scaffolds perform on real-world software engineering tasks:

• SWE-bench Leaderboards — the most widely cited benchmark for agentic coding. Models are evaluated on their ability to resolve real GitHub issues from open-source Python repos. The “Verified” split is the standard comparison point, though contamination concerns have motivated harder variants
• SWE-bench Pro — Scale AI’s harder benchmark (1,865 tasks across 41 repos). Top models that score 70%+ on SWE-bench Verified score only ~23% here
• SWE-Lancer — OpenAI’s benchmark based on 1,400+ real Upwork freelance tasks valued at $1M in payouts, ranging from $50 bug fixes to $32K feature implementations. Provides a natural difficulty gradient tied to real-world economics
• Terminal-Bench — evaluates agents on multi-step terminal workflows (not just code generation). Tests planning, execution, and recovery in sandboxed command-line environments
• Coding Agents Comparison — Artificial Analysis’s ongoing comparison with pricing breakdowns alongside benchmark scores
• Quantifying Infrastructure Noise in Agentic Coding Evals — Anthropic’s analysis showing that a 2-point leaderboard lead may reflect hardware differences rather than genuine capability gaps — important context for interpreting any benchmark

Caveat: Benchmarks measure specific capabilities under controlled conditions. Real-world performance depends heavily on your prompt quality, project structure, and CLAUDE.md configuration. Use benchmarks to track the field’s trajectory, not to pick a tool.

Security

• AI Coding Tools Exploded in 2025. The First Security Exploits Show What Could Go Wrong — Fortune’s reporting on the “IDEsaster” vulnerabilities found across Cursor, Copilot, Windsurf, and other tools
• Researcher Uncovers 30+ Flaws in AI Coding Tools — technical breakdown of the universal attack chains affecting major AI IDEs
• Security Flaws in Claude Code Risk Stolen Data, System Takeover — Check Point’s findings on Claude Code-specific CVEs, including hook injection and API key theft
• AI Agent Security Risks in 2026: A Practitioner’s Guide — practical guide to defending against prompt injection, credential theft, and MCP vulnerabilities

This is an area where best practices are being written in real time. What works today may be outdated in six months. Stay plugged into the communities above, and don’t assume any single tool or configuration is permanently “safe.”

The ruler analogy

Think of precision like the markings on a ruler. A high-precision ruler has markings at every millimeter — you can represent fine distinctions like 3.217 cm vs. 3.218 cm. A low-precision ruler might only have markings at each centimeter — you can still measure things, but 3.217 cm and 3.218 cm both round to 3 cm. You’ve lost the ability to distinguish them, but you need far less space to write down your measurement.

That’s exactly what happens with neural network weights. At FP32, a weight might be stored as 0.31415927. At FP16, it becomes 0.3142 — close, but not identical. At 4-bit, it gets mapped to one of only 16 possible values, like 0.3125. The question is whether those small differences matter for the model’s outputs. For most deep learning tasks, they don’t.

How floating-point numbers are stored

A floating-point number is stored in three parts:

• Sign bit (1 bit): positive or negative
• Exponent bits: control the range — how large or small the number can be (like the power in scientific notation)
• Fraction bits (aka mantissa): control the precision — how many significant digits you get

For example, FP32 uses 1 sign + 8 exponent + 23 fraction = 32 bits. FP16 cuts this to 1 + 5 + 10 = 16 bits. Fewer fraction bits means coarser rounding; fewer exponent bits means a narrower range of representable values. The table below summarizes the common formats:

Note that INT8 and NF4 are integer/discrete formats — they don’t have exponent and fraction parts at all. They can only represent a small, fixed set of values, and real-valued weights must be mapped onto those values (more on this in Part 3).

Key insight: precision controls memory per parameter

A model with 1 billion parameters requires:

• 4 GB at FP32 (4 bytes per param)
• 2 GB at FP16/BF16 (2 bytes per param)
• 1 GB at INT8 (1 byte per param)
• ~0.5 GB at 4-bit (0.5 bytes per param)

The number of parameters hasn’t changed — only how much memory each one occupies.

Let’s verify this with a real model.

Helper: Measure GPU memory

def get_gpu_memory_mb():
    """Return current GPU memory allocated in MB."""
    if torch.cuda.is_available():
        return torch.cuda.memory_allocated() / 1024**2
    return 0.0

def load_and_measure(model_name, dtype=None, quantization_config=None, label=""):
    """Load a model and report memory usage and parameter info."""
    gc.collect()
    torch.cuda.empty_cache()

    before = get_gpu_memory_mb()

    kwargs = {"device_map": "auto"}
    if dtype:
        kwargs["torch_dtype"] = dtype
    if quantization_config:
        kwargs["quantization_config"] = quantization_config

    model = AutoModelForCausalLM.from_pretrained(model_name, **kwargs)

    after = get_gpu_memory_mb()
    mem_used = after - before

    # Count parameters
    total_params = sum(p.numel() for p in model.parameters())
    total_elements = sum(p.nelement() for p in model.parameters())

    print(f"\n{'='*60}")
    print(f"  {label}")
    print(f"{'='*60}")
    print(f"  GPU memory used:     {mem_used:,.1f} MB")
    print(f"  model.parameters():  {total_params:,} (via numel())")
    print(f"  Expected params:     ~124,000,000 (GPT-2)")

    # Show dtypes present in model
    dtypes = set()
    for p in model.parameters():
        dtypes.add(str(p.dtype))
    print(f"  Parameter dtypes:    {dtypes}")

    return model, mem_used, total_params

FP32 (default)

model_name = "gpt2"

model_fp32, mem_fp32, params_fp32 = load_and_measure(
    model_name, dtype=torch.float32, label="FP32 (32-bit floating point)"
)
del model_fp32
gc.collect()
torch.cuda.empty_cache()

FP16 (half precision)

model_fp16, mem_fp16, params_fp16 = load_and_measure(
    model_name, dtype=torch.float16, label="FP16 (16-bit floating point)"
)
del model_fp16
gc.collect()
torch.cuda.empty_cache()

BF16 (bfloat16)

BF16 uses 16 bits like FP16, but allocates them differently (as shown in the table in Part 1). FP16 gives 10 bits to the fraction for finer precision, but only 5 bits to the exponent — which is why it caps out around 65,504 and can’t represent very small values. BF16 flips this tradeoff: it keeps the same 8-bit exponent as FP32 (giving it the same massive range), at the cost of only 7 fraction bits. In practice, this works well for deep learning — the range matters more than fine-grained precision, and BF16 avoids the overflow/underflow issues that can plague FP16 during training.

model_bf16, mem_bf16, params_bf16 = load_and_measure(
    model_name, dtype=torch.bfloat16, label="BF16 (bfloat16)"
)
del model_bf16
gc.collect()
torch.cuda.empty_cache()

Compare: precision vs. memory

print(f"\nMemory comparison (GPT-2, 124M params):")
print(f"  FP32:  {mem_fp32:,.1f} MB")
print(f"  FP16:  {mem_fp16:,.1f} MB")
print(f"  BF16:  {mem_bf16:,.1f} MB")
print(f"\nParameter count (should be identical):")
print(f"  FP32:  {params_fp32:,}")
print(f"  FP16:  {params_fp16:,}")
print(f"  BF16:  {params_bf16:,}")

At this point, the key takeaway should be clear: reducing precision halves memory, but the parameter count is unchanged. Every weight is still there — it just takes up less space.

Precision reduction vs. quantization: what’s the difference?

What we’ve done so far — loading a model in FP16 or BF16 instead of FP32 — is precision reduction (sometimes called “casting” or “downcasting”). It’s straightforward: each floating-point value is converted to a format with fewer bits, using standard IEEE rounding rules. The value 0.31415927 in FP32 becomes 0.3142 in FP16. There’s no special algorithm involved — it’s just rounding.

Quantization is fundamentally different. It doesn’t just round values to a lower-precision float — it maps them onto a small, discrete set of values (like the 256 integers in INT8, or just 16 values in NF4). This mapping requires decisions that simple rounding can’t make:

• What range of weight values should map to what integers? (This is called calibration.)
• Should all layers use the same mapping, or should each layer be calibrated separately?
• What do you do about outlier weights that fall far outside the typical range?

Different quantization algorithms answer these questions differently, and their choices directly affect how much quality you lose. That’s why quantization is a more involved process than just picking torch.float16 — it’s a compression technique with real engineering behind it.

Using bitsandbytes for quantization

bitsandbytes integrates directly with Hugging Face transformers, letting you apply these quantization algorithms at model load time with just a configuration flag. Let’s see the memory savings in practice.

8-bit quantization

bnb_config_8bit = BitsAndBytesConfig(load_in_8bit=True)

model_8bit, mem_8bit, params_8bit = load_and_measure(
    model_name, quantization_config=bnb_config_8bit, label="INT8 (8-bit via bitsandbytes)"
)
del model_8bit
gc.collect()
torch.cuda.empty_cache()

4-bit quantization (NF4)

bnb_config_4bit = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",           # normalized float 4-bit
    bnb_4bit_use_double_quant=True,       # further compress quantization constants
    bnb_4bit_compute_dtype=torch.float16  # compute in FP16 for speed
)

model_4bit, mem_4bit, params_4bit = load_and_measure(
    model_name, quantization_config=bnb_config_4bit, label="NF4 (4-bit via bitsandbytes)"
)

Compare all configurations

print(f"\n{'='*60}")
print(f"  Summary: GPT-2 (124M params) at different precisions")
print(f"{'='*60}")
print(f"  {'Config':<12} {'Memory (MB)':>14} {'numel()':>16}")
print(f"  {'-'*44}")
print(f"  {'FP32':<12} {mem_fp32:>14,.1f} {params_fp32:>16,}")
print(f"  {'FP16':<12} {mem_fp16:>14,.1f} {params_fp16:>16,}")
print(f"  {'BF16':<12} {mem_bf16:>14,.1f} {params_bf16:>16,}")
print(f"  {'INT8':<12} {mem_8bit:>14,.1f} {params_8bit:>16,}")
print(f"  {'NF4 (4-bit)':<12} {mem_4bit:>14,.1f} {params_4bit:>16,}")

What gets quantized (and what doesn’t)

When you load a model with bitsandbytes, only the large linear layer weight matrices are quantized. Smaller parameters — biases, layer normalization weights, and embedding layers — are kept in their original precision (typically FP16 or FP32). This is by design: these small parameters contribute little to total memory, and quantizing them would hurt quality disproportionately.

Why numel() is misleading for quantized parameters

For the parameters that are quantized, bitsandbytes packs multiple low-bit values into each byte:

• 8-bit: Each weight occupies 1 byte, stored as a uint8 tensor. numel() still returns the correct count here, since it’s one element per weight.
• 4-bit: Two weights are packed into a single byte, stored as a uint8 tensor of half the length.

When you call p.numel() on a 4-bit quantized parameter, PyTorch reports the number of elements in the storage tensor (the packed uint8 values), not the number of logical weights. Since two 4-bit values are packed into one uint8 element, numel() returns half the true count for those parameters. Combined with the non-quantized parameters (which report correctly), the total numel() across the model ends up somewhere between the true count and half — in GPT-2’s case, ~ 82M instead of ~ 124M.

Correctly counting parameters

To get the real parameter count, we need to check whether each parameter is a quantized bitsandbytes type and recover the original shape:

import bitsandbytes as bnb

def count_parameters_correct(model):
    """Count parameters correctly, handling bitsandbytes quantized layers."""
    total = 0
    quantized = 0
    non_quantized = 0

    for name, param in model.named_parameters():
        if hasattr(param, "quant_state"):
            # This is a bitsandbytes quantized parameter.
            # The original shape is stored in quant_state.
            original_numel = param.quant_state.shape.numel()
            total += original_numel
            quantized += original_numel
        else:
            total += param.numel()
            non_quantized += param.numel()

    return total, quantized, non_quantized

naive_count = sum(p.numel() for p in model_4bit.parameters())
correct_total, quantized_params, non_quantized_params = count_parameters_correct(model_4bit)

print(f"4-bit quantized GPT-2:")
print(f"  Naive numel() count:          {naive_count:,}")
print(f"  Correct parameter count:      {correct_total:,}")
print(f"    - Quantized (true count):   {quantized_params:,}  (numel reports ~{quantized_params // 2:,} due to packing)")
print(f"    - Non-quantized:            {non_quantized_params:,}  (numel reports correctly)")
print(f"  Expected (GPT-2):             ~124,000,000")
print(f"\n  Sanity check: {quantized_params // 2:,} (packed) + {non_quantized_params:,} (unquantized) ≈ {naive_count:,} (naive total) ✓")

What’s happening under the hood

Let’s peek at an individual layer to see the difference between the stored tensor shape and the logical weight shape:

for name, param in model_4bit.named_parameters():
    if hasattr(param, "quant_state"):
        print(f"Layer: {name}")
        print(f"  Storage tensor shape: {param.shape}")
        print(f"  Storage dtype:        {param.dtype}")
        print(f"  numel() reports:      {param.numel():,}")
        print(f"  Original shape:       {param.quant_state.shape}")
        print(f"  Original numel:       {param.quant_state.shape.numel():,}")
        print()
        break  # just show one example

This confirms that quantization doesn’t remove parameters — it repacks them into a more compact representation. The model’s architecture and logical weight count are unchanged, but the storage is compressed.

Summary

The bottom line: quantization does not remove parameters. It changes how they’re stored. Always use quant_state or measure GPU memory directly if you want an accurate picture of a quantized model.

Quantization is primarily an inference technique

Quantization shines at inference time. Training requires high-precision gradients to make stable updates to model weights, and aggressive quantization (8-bit or below) introduces too much noise for standard backpropagation to work well. For this reason, most models are trained at FP32, BF16, or with mixed-precision strategies (FP16 compute with FP32 accumulation), and then quantized after training for deployment.

The notable exception is QLoRA (Dettmers et al., 2023), which freezes a 4-bit quantized base model and trains only small low-rank adapter (LoRA) layers in higher precision. This makes it possible to fine-tune a 65B-parameter model on a single 48GB GPU — but the base weights themselves are never updated in low precision.

A bigger model at lower precision often beats a smaller model at full precision

One of the most practical insights from quantization research: you can often get better results by running a larger model at 4-bit than a smaller model at FP16, using the same GPU memory. For example:

• A 70B model at 4-bit (~ 35 GB) can fit on a single 48GB GPU and typically outperforms a 13B model at FP16 (~ 26 GB) on reasoning and knowledge benchmarks.
• A 13B model at 4-bit (~ 6.5 GB) fits comfortably on a 12GB consumer GPU and often outperforms a 7B model at FP16 (~ 14 GB, which wouldn’t even fit).

The rule of thumb: spend your memory budget on more parameters first, then reduce precision to fit. A 4-bit model loses a small amount of quality from quantization, but it gains far more from having access to more learned knowledge and capacity.

Speed and cost: quantization isn’t just about fitting

Quantization isn’t just about fitting a model onto your GPU — it also makes inference faster. Lower-precision operations use less memory bandwidth, and for small batch sizes (which are common in interactive applications), memory bandwidth is often the bottleneck. So a 4-bit model doesn’t just use ~8x less memory than FP32 — it can also generate tokens noticeably faster.

This creates a practical consideration:

• The question worth asking is not just “which model is most accurate?” but “which model gives me acceptable quality at the speed and cost I need?”
• For batch applications (processing thousands of documents), the speed improvement from quantization can cut costs significantly.
• For interactive applications (chatbots, coding assistants), faster token generation directly improves user experience.

How low can you go?

Research suggests 4-bit is the practical sweet spot for inference:

• Dettmers & Zettlemoyer (2023) ran over 35,000 quantization experiments and found that 4-bit precision is nearly universally optimal when trading off total model bits against zero-shot accuracy. At 3-bit, quality degrades sharply.
• 8-bit quantization (via LLM.int8()) is effectively lossless for most models — it’s a safe default when memory is tight but you don’t want to risk any quality loss.
• GPTQ (Frantar et al., 2023) demonstrated that one-shot weight quantization to 3-4 bits is feasible even for 175B-parameter models with negligible accuracy loss, enabling single-GPU inference for models that otherwise require multiple GPUs.

A recent study on Scaling Laws for Precision (Kumar, Ankner et al., 2024) adds important nuance: the quality degradation from post-training quantization grows as models are trained on more data. A model trained to its full data budget may be more sensitive to aggressive quantization than one that was undertrained. This means the “safe” bit-width may shift upward as foundation models continue to scale their training data.

Decision flowchart

When deciding whether and how to quantize, work through these questions:

1. Does the model fit on your GPU at FP16/BF16? If yes, start there — no quantization needed unless you want faster inference.
2. Is this for training or inference?
   • Training from scratch: Use BF16 (or FP32 if your GPU lacks BF16 support). Don’t quantize.
   • Fine-tuning: If the full model doesn’t fit, use QLoRA (4-bit base + FP16 adapters).
   • Inference: Continue to step 3.
3. How much quality loss can you tolerate?
   • None: Use INT8. It’s effectively lossless for most models.
   • Minimal, with significant memory savings: Use 4-bit NF4 with double quantization.
   • You need extreme compression: Try 3-bit, but benchmark on your specific task — quality loss at 3-bit is sharp and task-dependent.
4. Could you run a larger model by quantizing more aggressively? Often the answer is yes. A 70B model at 4-bit typically beats a 13B model at 16-bit.

Quick reference

Cloud vs. university HPC clusters

Most universities offer shared HPC clusters with GPUs. These are excellent resources — but they have tradeoffs worth understanding:

The short version: use your university cluster when it has the hardware you need and the queue isn’t blocking you. Use the cloud when you need hardware your cluster doesn’t have, need to scale beyond what the queue allows, or need a specific software environment you can’t easily get on-campus. Many researchers use both — develop and test on HPC, then scale to cloud for large experiments or specialized hardware.

When does model size justify cloud compute?

Not every model needs cloud hardware. Here’s a rough guide:

CHTC’s GPU Lab covers more than you might think. The GPU Lab includes A100s (40 and 80 GB), H100s (80 GB), and H200s (141 GB) — enough VRAM to run inference on models up to ~70B parameters with quantization, or to fine-tune smaller models on a single high-memory GPU. For many UW researchers, this hardware handles “large model” workloads without needing cloud. Note that CHTC GPUs are not NVLink-connected, so multi-GPU parallelism is limited to methods that don’t require fast inter-GPU communication. Jobs have time limits (12 hrs for short, 24 hrs for medium, 7 days for long jobs), so plan your runs accordingly.

Cloud becomes the clear choice when you need NVLink multi-GPU or multi-node setups for frontier-scale training or inference, long-running services like RAG applications or model endpoints that need to stay up beyond HPC job time limits, or when queue wait times are blocking a deadline.

LLM APIs: skip the infrastructure entirely

For many GenAI tasks, you don’t need to provision GPUs at all. Services like the OpenAI API, Google’s Vertex AI, and Amazon Bedrock let you call frontier models (GPT-4o, Gemini, Claude, etc.) with a simple API request — no GPU provisioning, no model hosting. LLM API calls cost fractions of a cent each and are often the fastest, most cost-effective path. See GenAI at UW-Madison for available services.

A note on cloud costs

Cloud computing is not free, but it’s worth putting costs in context:

• Hardware is expensive and ages fast. A single A100 GPU costs ~$15,000 and is outdated within a few years. Cloud lets you rent the latest hardware by the hour.
• You pay only for what you use. Stop a VM and the meter stops — valuable for bursty research workloads. A single T4 GPU instance runs ~$1–3/hr. Fine-tuning a small model on a moderate dataset might cost $10–50.
• Managed services save development time. You don’t have to write scheduling logic, package custom containers, or maintain orchestration infrastructure — managed ML platforms handle that plumbing so you can focus on the ML.
• Budgets and alerts keep you safe. All three platforms offer billing dashboards and budget alerts to prevent surprise bills.

The key habit: choose the right machine size, stop resources when idle, and monitor spending.

Lower overhead (Cloud Computing Pilot)

UW-Madison normally adds 55.5% in overhead (formally called “F&A” or “facilities & administrative costs”) to cloud expenses on grants. The Cloud Computing Pilot cuts that to 26% when you use a UW-provisioned cloud account. In practice, that means for every $10,000 in cloud spending, you’ll pay ~$2,600 in overhead instead of ~$5,550 — a savings of about $2,950.

• Applies to new proposals and awards (including new funding increments).
• You must use a UW cloud account — costs paid via purchasing card or personal accounts are charged the full 55.5%.
• RSP provides budget templates to help you plan proposals with the reduced rate.
• Contact RSP with questions about grant compliance.

NIH STRIDES Initiative

If you have NIH funding specifically, you can get additional cloud discounts on top of the standard UW rates through the STRIDES Initiative. STRIDES covers AWS, GCP, and Azure:

• Discounted pricing on cloud services, layered on top of UW’s institutional rates.
• Professional service consultations and technical support from STRIDES partners.
• No data or configuration changes needed — the UW cloud team can transition you in or out at any time.

Research credits

All three cloud providers offer credit programs for academic researchers:

All three programs are rolling applications. You’ll need a research proposal describing your intended cloud usage and the specific services you plan to use.

Grants for social impact & sustainability research

The major cloud providers also offer larger grants for research focused on public good — sustainability, environmental science, public health, education, and underserved communities:

• Google: The Google.org Impact Challenge: AI for Science awards $500K–$3M for projects using AI to tackle scientific challenges, with a specific focus area on climate resilience and environmental science (biodiversity, agriculture, oceans). Applications open through April 17, 2026.
• AWS: The AWS Imagine Grant provides up to $200K in unrestricted funding plus AWS credits to nonprofits and research organizations working on social impact. Past winners include sustainability, public health, and underserved community projects. The 2026–2027 cycle opens spring 2026.
• Microsoft: The AI for Good Lab runs open calls awarding Azure credits and scientific collaboration for projects in sustainability, public health, education, and human rights. Academic institutions are eligible. Microsoft also offers free access to petabytes of environmental data through the Planetary Computer.

Free cloud training

Each platform offers free, self-paced training to help you get started:

• GCP: UW-Madison has a limited number of seats for Google Cloud Skills Boost — reach out to the Public Cloud Team at cloud-services@cio.wisc.edu to request access.
• AWS: AWS Skill Builder offers 600+ free courses covering compute, ML, and more.
• Azure: Microsoft Learn provides free, structured learning paths for Azure services.

The self-driving car analogy

The trajectory of SWE-bench scores is reminiscent of autonomous driving predictions circa 2015–2017, when rapid progress on structured benchmarks led many companies to predict full autonomy was just a year or two away. A decade later, the long tail of edge cases turned out to be the hardest part.

Similarly, while the pace of improvement on SWE-bench is genuinely impressive, the remaining 25–30% of unresolved issues — and the much larger space of tasks not captured by the benchmark — may prove disproportionately difficult. Benchmarks measure a specific, well-defined slice of capability, and the gap between benchmark performance and reliable, general-purpose software engineering likely remains significant.

Understand the risk before you start

When you launch Claude Code from the CLI, it has access to your current working directory — but it is not strictly confined to it. Without sandboxing enabled, Claude can read, modify, create, or delete files anywhere your user account has access, including parent directories, your home folder, and system files. Independent testing has confirmed that without sandboxing, Claude can create files in parent directories above the working directory without any special prompt.

This means a poorly worded prompt, an agentic loop, or a prompt injection attack embedded in a file Claude reads could cause changes in places you didn’t intend.

Practical recommendations:

• Avoid running Claude Code on machines where restricted or sensitive data is stored. Sandboxing restricts writes but not reads — Claude can still read files anywhere on the machine and send their contents to Anthropic’s servers as part of a prompt. UW does not yet have a data-use agreement with Anthropic (see warning above), so even cloud-routed usage sends your prompts to Anthropic’s infrastructure.
• If you must run on a machine with sensitive files, add deny rules (see below) to block read access to sensitive paths. But note that Read deny rules are best-effort — allowed Bash commands like cat can still read denied files. Denying both Read and the relevant Bash patterns together provides stronger protection (see this security deep-dive for details).
• Test only within isolated Git repositories, with no sensitive data present in the local repo.
• Always enable sandboxing (Section 8) — it restricts writes to your project directory and adds network isolation at the OS level. It’s available on macOS (built-in) and Linux/WSL2 (requires bubblewrap). Native Windows does not yet support it.

Always launch from your project directory

cd C:\Users\you\projects\my-project
claude

cd ~/projects/my-project
claude

Claude Code defaults to operating in the directory where it’s launched. Starting from C:\ or / or your home directory gives it an unnecessarily broad scope.

Use Git as your safety net

Always work inside a Git repository. This is your most important protection:

• You can instantly see what changed with git diff.
• You can revert any unwanted changes with git checkout . or git stash.
• Claude Code itself is Git-aware and will generally respect repository boundaries.

Before starting a Claude Code session, make sure your working tree is clean (commit or stash pending changes). That way, anything Claude does can be cleanly reviewed or rolled back.

Use the default permission mode

Out of the box, Claude Code asks for your approval before running most commands. Do not change this unless you understand the implications. Specifically:

• Do NOT use --dangerously-skip-permissions unless you’re in an isolated container/VM. This flag bypasses all safety checks.
• Do NOT set the mode to bypassPermissions in settings.

Understand how permissions actually work

In the default mode, Claude Code already asks for your approval before running most commands that modify your system (file writes, bash commands, network requests, etc.). Read-only operations like viewing files generally run without prompting.

You can customize this behavior with three types of rules in settings.json:

• deny — Hard block. Claude cannot use the tool, period. You won’t even be asked. Deny rules always win over allow rules.
• allow — Auto-approve. Claude can use the tool without asking you first. This is a convenience shortcut — it skips the approval prompt for things you trust.
• ask — Force a prompt, even if something else would auto-approve it.

You do NOT need to add allow rules for Claude Code to work. Without any custom rules, it will simply ask you to approve each action as it comes up. The default behavior is already safe — deny rules are the ones that add protection, while allow rules just reduce the number of prompts you see.

Add deny rules to protect sensitive areas

Add a permissions block to your ~/.claude/settings.json file (the same file you edited in Section 5). Think of deny rules as guardrails — they block Claude from accessing sensitive paths regardless of what it tries to do.

These examples block: SSH keys, .env files, cloud credentials (~/.config/gcloud/ and ~/.aws/), destructive commands (rm -rf), and network tools (curl, wget). Replace endemann with your actual username before pasting into settings.json.

{
  "permissions": {
    "deny": [
      "Read(//Users/endemann/.ssh/**)",
      "Edit(//Users/endemann/.ssh/**)",
      "Read(//Users/endemann/.env)",
      "Edit(//Users/endemann/.env)",
      "Read(//Users/endemann/.config/gcloud/**)",
      "Edit(//Users/endemann/.config/gcloud/**)",
      "Read(//Users/endemann/.aws/**)",
      "Edit(//Users/endemann/.aws/**)",
      "Bash(rm -rf *)",
      "Bash(curl:*)",
      "Bash(wget:*)"
    ]
  }
}

{
  "permissions": {
    "deny": [
      "Read(//C:/Users/endemann/.ssh/**)",
      "Edit(//C:/Users/endemann/.ssh/**)",
      "Read(//C:/Users/endemann/.env)",
      "Edit(//C:/Users/endemann/.env)",
      "Read(//C:/Users/endemann/.config/gcloud/**)",
      "Edit(//C:/Users/endemann/.config/gcloud/**)",
      "Read(//C:/Users/endemann/.aws/**)",
      "Edit(//C:/Users/endemann/.aws/**)",
      "Bash(rm -rf *)",
      "Bash(curl:*)",
      "Bash(wget:*)"
    ]
  }
}

{
  "permissions": {
    "deny": [
      "Read(//home/endemann/.ssh/**)",
      "Edit(//home/endemann/.ssh/**)",
      "Read(//home/endemann/.env)",
      "Edit(//home/endemann/.env)",
      "Read(//home/endemann/.config/gcloud/**)",
      "Edit(//home/endemann/.config/gcloud/**)",
      "Read(//home/endemann/.aws/**)",
      "Edit(//home/endemann/.aws/**)",
      "Bash(rm -rf *)",
      "Bash(curl:*)",
      "Bash(wget:*)"
    ]
  }
}

Add allow rules to reduce prompt fatigue

These also go in ~/.claude/settings.json, inside the same permissions block as your deny rules. By default, Claude Code asks “Do you want to proceed?” before every file write, shell command, and git operation. This gets old fast — in a typical session you might approve the same git add, git commit, and git push sequence a dozen times. You can add allow rules so Claude runs trusted commands without asking:

Minimal — just stop the git and test prompts:

{
  "permissions": {
    "allow": [
      "Bash(git add *)",
      "Bash(git commit *)",
      "Bash(git push *)",
      "Bash(git checkout *)",
      "Bash(git log *)",
      "Bash(git diff *)",
      "Bash(git status)",
      "Bash(git rm *)",
      "Bash(gh *)",
      "Bash(python *)",
      "Bash(pytest *)"
    ]
  }
}

More aggressive — also auto-approve file edits:

{
  "permissions": {
    "allow": [
      "Edit",
      "Write",
      "Bash(git *)",
      "Bash(gh *)",
      "Bash(python *)",
      "Bash(pytest *)",
      "Bash(npm *)",
      "Bash(pip *)"
    ]
  }
}

You don’t have to add any allow rules — the default “ask before doing” behavior is safe, especially when you’re starting out. But if you find yourself mindlessly hitting “Yes” on every prompt, that’s a sign you should either add allow rules or enable sandboxing.

Check your current permissions

Run /permissions inside a Claude Code session at any time to see what rules are active and which settings file they came from. This is especially helpful to verify your deny rules are loaded correctly.

Use project-level settings for shared repos

Claude Code reads two settings files, and merges them:

How merging works: Claude Code combines both files. If either file denies something, it’s denied — project-level deny rules apply to everyone, even if their personal settings are more permissive. You don’t have to choose one or the other; most teams use both.

The project-level file lives inside your Git repo, so you can commit it and every collaborator automatically inherits the same rules:

{
  "permissions": {
    "deny": [
      "Read(./.env)",
      "Read(./secrets/**)",
      "Bash(curl:*)",
      "Bash(wget:*)"
    ]
  }
}

Interactive safeguards during a session

• Press Esc at any time to interrupt Claude Code mid-operation.
• Press Shift+Tab to cycle through permission modes during a session.
• Use /permissions to view and manage active permissions.
• Review every command before approving it — especially rm, mv, file writes outside your project, and anything involving sudo.

Permission mode cheat sheet

Further reading on permissions & security

• Official permissions docs — full reference for rule syntax, tool names, managed policies
• Official sandboxing docs — setup, configuration, and security model for OS-level isolation
• Anthropic engineering blog on sandboxing — explains the design rationale and how filesystem + network isolation work together
• Security deep-dive by Pete Freitag — independent analysis of how permissions actually work under the hood, including gotchas and bypasses
• Permission modes guide — when to use each mode, with workflow examples

Typical costs to expect

According to Anthropic’s own documentation, the average Claude Code session costs about $6 per developer per day, with 90% of users staying under $12/day. However, this is an average — complex tasks, long sessions, or agentic loops can blow past this easily. Monthly costs with Sonnet typically run $100–$200/developer, but there’s large variance.

Use the /cost command inside Claude Code at any time to see your current session’s token usage and estimated cost:

/cost

Understand what drives costs

Token costs scale with context size — the more context Claude processes, the more you pay. Key cost drivers:

• Long conversations: Claude re-processes the entire conversation history with each message. A session that’s been going for hours costs more per-message than a fresh one.
• Large codebases: If Claude reads many files to understand your project, that’s all input tokens.
• Extended thinking: Enabled by default with a budget of ~32K tokens. Thinking tokens are billed as output tokens (the expensive kind). For simple tasks, this is overkill.
• Agentic loops (see below): The single biggest risk for runaway costs.

Agentic loops — the #1 cost risk

Real-world examples from the community:

• One user reported a bug where an auto-compacting loop consumed 108 million tokens in a single day ($64–$78), compared to their normal 12–68M tokens/day.
• Runaway spikes of $235+ over a 4-day period were documented from that same loop bug.
• Version updates have occasionally caused 4x faster token consumption on the same tasks.

How to protect yourself:

• Watch for signs: If Claude seems to be reading the same files over and over, or your /cost jumps unusually fast, press Esc immediately to interrupt.
• Use /compact to compress long conversations before they bloat.
• Use /clear when switching to unrelated tasks — don’t let stale context accumulate.
• Be specific in prompts. Vague requests like “improve this codebase” trigger broad scanning. “Add input validation to the login function in auth.ts” is much cheaper.
• Use plan mode (Shift+Tab) before expensive operations. Claude outlines its approach for your approval before writing code, preventing costly rework.
• Start with Sonnet, not Opus. Only escalate to Opus for genuinely complex reasoning tasks. Opus costs significantly more per token.

Set up budget alerts (do this NOW)

Budget alerts won’t automatically stop spending, but they’ll email you when you’re approaching a threshold. Set this up before your first Claude Code session.

Check your billing console daily

Make it a habit to check your cloud billing daily while actively using Claude Code. Look for:

• Unexpected spikes in AI/ML charges
• Daily costs that exceed your expectations
• Any charges from services you didn’t intentionally use

Cost-saving tips

Pricing reference

For the latest Claude model pricing, see:

• Anthropic Pricing — base token rates by model, including cache and batch discounts
• Claude Code Cost Management — Anthropic’s official guide to tracking and reducing Claude Code costs
• Vertex AI Generative AI Pricing — Google’s per-token rates (regional endpoints carry a 10% premium over global)
• Amazon Bedrock Pricing — AWS per-token rates for Claude models

Training policy

Whether your code is used for model training depends on your account type:

If you’re accessing Claude Code through a UW-Madison cloud account (Vertex AI or Bedrock), your usage falls under Anthropic’s commercial terms — your code is not used for training. However, note that UW-Madison does not yet have a direct data-use agreement with Anthropic. The no-training guarantee comes from Anthropic’s standard commercial terms, not from a UW-negotiated agreement. This distinction matters for restricted or regulated data — see the data sensitivity warning at the top of this guide.

Data retention

Claude Code also caches sessions locally on your machine for up to 30 days to enable session resumption (configurable).

Telemetry and error reporting

Claude Code sends operational metrics and error reports by default when using the direct Claude API:

Common issues (both providers)

claude command not found:

Run diagnostics:

claude doctor

This checks your installation, authentication, and configuration for common issues.

In-session commands (all providers)

/status        # Confirm your cloud provider is active
/cost          # Check current session token usage and cost
/compact       # Compress conversation context to save tokens
/clear         # Wipe context when switching tasks
/permissions   # View and manage active permission rules
/sandbox       # Check or enable OS-level sandboxing
Esc            # Interrupt Claude mid-operation
Shift+Tab      # Cycle through permission modes

Set up your CLAUDE.md

CLAUDE.md is a markdown file in your project root that gives Claude persistent context — build commands, test commands, architectural conventions, anything Claude can’t infer from the code alone. It loads automatically at the start of every session.

Run /init inside Claude Code to generate a starter file, but treat the output as a draft, not a finished product. Manually curate it down to only what Claude actually needs. Research from ETH Zurich (Banerjee et al., 2025) found that LLM-generated context files can actually decrease agent performance compared to no context file at all — the auto-generated content adds noise that dilutes the instructions that matter.

Pick the right model for the task

Your settings.json sets a default model, but you can switch mid-session with /model. A general rule of thumb:

Start with Sonnet — it handles the vast majority of tasks well. Escalate to Opus only when you notice Sonnet struggling with complex reasoning. Use Haiku for high-volume, low-complexity work where cost matters.

Build good habits early

• Commit before you start. A clean Git state means you can always git diff to see what Claude changed, or git checkout . to revert.
• Be specific. “Add input validation to the login() function in auth.py” is cheaper and more reliable than “improve the auth module.”
• Use /compact and /clear to manage context. Long sessions get expensive and Claude’s attention degrades as context grows (Liu et al., 2024).
• Press Esc the moment something looks wrong. Don’t let an agentic loop run up your bill — see Section 9.

Getting started

• Official Tutorial

• code snippet (this code loads audio as a tensor and resamples it so every file has the same sample rate):

  # wav -> log-mel spectrogram tensor (which would be model input)
  import torchaudio
  
  w, sr = torchaudio.load("path/to/audio.wav")
  w = w.mean(0, keepdim=True) if w.size(0) > 1 else w
  if sr != 22050: w = torchaudio.transforms.Resample(sr, 22050)(w); sr = 22050
  
  mel = torchaudio.transforms.MelSpectrogram(sample_rate=sr, n_fft=2048, hop_length=512, n_mels=128)
  X_db = torchaudio.transforms.AmplitudeToDB(stype="power")(mel(w))
  print(X_db.shape)  # shape would be (1, 128, T)  

High-level tips for effective use

• Optimization: precompute log-mels for quicker training
• Memory Management: use small-ish n_mels and hop_length and also batch by time frames
• Common Pitfalls: a common mistake is having inconsistent sample rates and hop lengths, so you should make sure to keep them the same in training and inference

Related libraries & tools

• librosa: offers extra MIR utilities, like chroma and beat tracking
• pretty_midi: turns frame-wise predictions into MIDI files for eventual listening/evaluation

Step 1: Get Your GitHub Token

The GDD API uses GitHub OAuth for authentication. You need to generate a personal access token.

1. Go to your GitHub Settings.
2. Navigate to Developer settings > Personal access tokens > Tokens (classic).
3. Click Generate new token (classic). Give it a descriptive note (e.g., “GeoDeepDive API”).
4. Select the public_repo scope. This is sufficient.
5. Click Generate token and copy the token immediately (you won’t see it again!).

Step 2: Set Up Your Python Environment

We’ll use the requests library to make HTTP calls. Let’s install it:

!pip install requests

Step 3: Configure Your Authentication

Now, let’s set up your authentication. Replace the placeholders with your actual GitHub credentials:

import requests
import json

# Replace these with your actual GitHub credentials
GITHUB_USERNAME = "YourGitHubUsername" 
GITHUB_TOKEN = "YOUR_GITHUB_TOKEN"  # Replace with the token you generated

print("Authentication configured successfully!")

Step 4: Query the GeoDeepDive API

Let’s search for documents mentioning the mineral “stishovite”:

# The public endpoint for the GDD API
url = "https://geodeepdive.org/api/articles"

# The parameters for our query. We want sentences about 'stishovite'
params = {
    "term": "stishovite",   # The word or phrase to search for
    "full_results": True,   # Get full details, including sentences
    "sentences": True       # Include the sentences in the response
}

# Make the GET request to the API with authentication
response = requests.get(url, params=params, auth=(GITHUB_USERNAME, GITHUB_TOKEN))

# Check if the request was successful
if response.status_code == 200:
    data = response.json()
    print(f" Found {data['success']['total']} documents mentioning 'stishovite'.\n")
    
    # Loop through the first few documents and print relevant sentences
    for i, doc in enumerate(data['success']['data'][:3]):  # Look at first 3 docs
        print(f" Document {i+1}: {doc['_gddid']}")
        print(f"   Title: {doc.get('title', 'No title available')}")
        print("   Sentences found:")
        
        stishovite_sentences = [s for s in doc['sentences'] if 'stishovite' in s['text'].lower()]
        
        for j, sentence in enumerate(stishovite_sentences[:2]):  # Show first 2 sentences per doc
            print(f"     {j+1}. {sentence['text']}")
        
        print(f"   Total sentences with 'stishovite': {len(stishovite_sentences)}")
        print("\n" + "─" * 80 + "\n")
else:
    print(f" Error: {response.status_code}")
    print(response.text)

Step 5: Advanced Query - Filter by Journal

Let’s try a more specific query to find papers in specific journals:

# Search for stishovite in specific journals
advanced_params = {
    "term": "stishovite",
    "journal": "science,nature,geology",  # Filter by journal names
    "full_results": True,
    "sentences": True,
    "limit": 5  # Limit to 5 results
}

advanced_response = requests.get(url, params=advanced_params, auth=(GITHUB_USERNAME, GITHUB_TOKEN))

if advanced_response.status_code == 200:
    advanced_data = advanced_response.json()
    print(f"🔍 Found {advanced_data['success']['total']} documents in specified journals.")
    
    if advanced_data['success']['total'] > 0:
        print("\n📊 Journal distribution:")
        journals = {}
        for doc in advanced_data['success']['data']:
            journal = doc.get('journal', 'Unknown')
            journals[journal] = journals.get(journal, 0) + 1
        
        for journal, count in journals.items():
            print(f"   {journal}: {count} documents")
    else:
        print("No documents found in the specified journals.")
else:
    print(f" Advanced query error: {advanced_response.status_code}")

Step 6: Export Results (Optional)

Let’s export the results to a JSON file for further analysis:

import json
from datetime import datetime

# Export the results
if response.status_code == 200:
    export_data = {
        "query": "stishovite",
        "execution_date": datetime.now().isoformat(),
        "total_documents": data['success']['total'],
        "sample_documents": data['success']['data'][:5]  # First 5 documents
    }
    
    with open('geodeepdive_results.json', 'w') as f:
        json.dump(export_data, f, indent=2)
    
    print("💾 Results exported to 'geodeepdive_results.json'")
    
    # Show a preview
    print("\n📋 Preview of exported data:")
    print(f"Total documents: {export_data['total_documents']}")
    print(f"Sample size: {len(export_data['sample_documents'])}")