> ## Documentation Index
> Fetch the complete documentation index at: https://mintlify.com/p-e-w/heretic/llms.txt
> Use this file to discover all available pages before exploring further.
# Configuration Overview
> Learn how to configure Heretic using config files and CLI flags
## Configuration Methods
Heretic supports multiple ways to configure its behavior, providing flexibility for different workflows:
* **CLI flags** - Pass options directly on the command line
* **Configuration files** - Use TOML files for persistent settings
* **Environment variables** - Set options via `HERETIC_` prefixed variables
These configuration sources are processed in order of priority (highest to lowest):
1. CLI flags (highest priority)
2. Environment variables with `HERETIC_` prefix
3. TOML configuration file
CLI flags will override any settings in your config file or environment variables.
## Using Configuration Files
### File Location and Naming
Heretic looks for a configuration file named `config.toml` in the directory where you run the command. The configuration file uses the TOML format, which is human-readable and easy to edit.
```bash Terminal theme={null}
# Run heretic with config.toml in current directory
heretic meta-llama/Llama-3.1-8B-Instruct
```
```toml config.toml theme={null}
# Your configuration options here
batch_size = 64
max_response_length = 150
quantization = "bnb_4bit"
```
### Creating Your First Config File
Heretic includes a default configuration file that you can use as a starting point. Copy the `config.default.toml` from the repository and rename it to `config.toml`:
```bash theme={null}
# Rename the default config to use it
cp config.default.toml config.toml
```
Then edit `config.toml` to customize the settings for your needs.
## CLI Flags vs Config Files
### When to Use CLI Flags
CLI flags are ideal for:
* **One-time adjustments** - Testing different settings without modifying your config file
* **Scripting** - Automating heretic runs with different parameters
* **Quick experiments** - Trying out a single option change
```bash Quick Test theme={null}
# Try 4-bit quantization without editing config
heretic Qwen/Qwen3-4B-Instruct-2507 --quantization bnb_4bit
```
```bash Evaluation theme={null}
# Evaluate a model with custom response length
heretic google/gemma-3-12b-it \
--evaluate-model p-e-w/gemma-3-12b-it-heretic \
--max-response-length 200
```
### When to Use Config Files
Configuration files are ideal for:
* **Consistent workflows** - Maintaining standard settings across multiple runs
* **Complex configurations** - Managing many options at once
* **Documentation** - Keeping a record of settings that worked well
* **Custom datasets** - Configuring dataset specifications with all their options
## Basic Configuration Example
Here's a basic configuration that enables 4-bit quantization, uses a moderate batch size, and extends the response length:
```toml config.toml theme={null}
# Model loading configuration
quantization = "bnb_4bit" # Reduce VRAM usage
device_map = "auto" # Automatically distribute across GPUs
# Performance tuning
batch_size = 32 # Process 32 sequences at once
max_batch_size = 64 # Don't exceed 64 when auto-detecting
# Evaluation settings
max_response_length = 150 # Generate up to 150 tokens
print_responses = true # Show what the model generates
# Optimization parameters
n_trials = 150 # Run 150 optimization trials
n_startup_trials = 50 # Use 50 trials for exploration
```
```bash Usage theme={null}
# Run with the configuration above
heretic meta-llama/Llama-3.1-8B-Instruct
```
Start with the default configuration and adjust only the settings you need to change. The defaults are well-tuned for most use cases.
## Configuration Sections
Heretic's configuration is organized into logical sections:
Configure dtypes, quantization, device mapping, and memory limits
Control the optimization process, batch sizes, and abliteration parameters
Set up datasets, refusal markers, and evaluation prompts
## Viewing All Options
To see all available configuration options and their descriptions:
```bash theme={null}
heretic --help
```
This displays all CLI flags with their descriptions, default values, and data types.
Option names in CLI flags use kebab-case (e.g., `--max-response-length`), while the same options in TOML files use snake\_case (e.g., `max_response_length`).
## Environment Variables
You can also set configuration options using environment variables with the `HERETIC_` prefix:
```bash theme={null}
# Set configuration via environment variables
export HERETIC_QUANTIZATION=bnb_4bit
export HERETIC_MAX_RESPONSE_LENGTH=200
export HERETIC_BATCH_SIZE=32
# Run heretic (will use environment variables)
heretic Qwen/Qwen3-4B-Instruct-2507
```
Environment variables use uppercase with underscores (e.g., `HERETIC_MAX_RESPONSE_LENGTH`).
## Example: Alternative Use Case
Heretic can also be configured to remove "slop" (purple prose and clichés) from creative writing models. Here's an example based on `config.noslop.toml`:
```toml config.noslop.toml theme={null}
max_response_length = 300
refusal_markers = [
"Eldoria", "Lumina", "ethereal", "celestial",
"radiant", "crimson", "velvet", "twilight",
"symphony", "tapestry", "ancient", # ... more markers
]
system_prompt = "You are a professional writer."
[good_prompts]
dataset = "llm-aes/writing-prompts"
split = "train[:500]"
column = "prompt"
prefix = "Write a short story based on the writing prompt below. Avoid literary cliches, purple prose, and flowery language.\n\nWriting prompt:"
[bad_prompts]
dataset = "llm-aes/writing-prompts"
split = "train[:500]"
column = "prompt"
prefix = "Write a short story based on the writing prompt below. Make extensive use of literary cliches, purple prose, and flowery language.\n\nWriting prompt:"
```
```bash Usage theme={null}
# Copy and use the noslop configuration
cp config.noslop.toml config.toml
heretic Qwen/Qwen3-4B-Instruct-2507
```
This demonstrates how Heretic's configuration system can be adapted for different purposes beyond censorship removal.
## Next Steps
Configure how models are loaded into memory
Fine-tune the abliteration optimization process
Set up datasets and refusal detection