Skip to content

Configuration Overview

The Fine-Tune Pipeline uses a TOML-based configuration system that allows you to customize every aspect of the training, inference, and evaluation processes. All configuration is centralized in the config.toml file.

Configuration Structure

The configuration file is organized into three main sections:

[fine_tuner]     # Fine-tuning configuration
[inferencer]     # Inference configuration  
[evaluator]      # Evaluation configuration

Configuration Loading

The pipeline automatically loads configuration from config.toml in the project root. The configuration system supports:

  • Environment Variable Substitution: Use ${VARIABLE_NAME} syntax
  • Null Values: Use "null" string for optional parameters
  • Type Conversion: Automatic conversion of strings to appropriate types

Example with Environment Variables

[fine_tuner]
base_model_id = "${BASE_MODEL_ID}"
wandb_project_name = "${WANDB_PROJECT}"

Common Configuration Patterns

Development vs Production

[fine_tuner]
base_model_id = "unsloth/Qwen2.5-0.5B-Instruct-bnb-4bit"
epochs = 1
device_train_batch_size = 2
push_to_hub = false

[fine_tuner]
base_model_id = "unsloth/Llama-3.2-3B-Instruct-bnb-4bit"
epochs = 5
device_train_batch_size = 8
push_to_hub = true

Memory-Constrained Training

For systems with limited GPU memory:

[fine_tuner]
load_in_4bit = true
device_train_batch_size = 1
grad_accumulation = 16
max_sequence_length = 1024
use_gradient_checkpointing = "unsloth"

High-Performance Training

For systems with ample resources:

[fine_tuner]
load_in_4bit = false
device_train_batch_size = 16
grad_accumulation = 2
max_sequence_length = 4096
packing = true

Configuration Validation

The pipeline validates configuration at startup and will report errors for:

  • Missing required fields
  • Invalid data types
  • Conflicting parameter combinations
  • Invalid model or dataset IDs

Common Validation Errors

❌ base_model_id is required
❌ epochs must be a positive integer
❌ learning_rate must be between 0 and 1
❌ Cannot use both load_in_4bit and load_in_8bit

Environment-Specific Configurations

Using Multiple Config Files

You can override the default config file:

from app.config_manager import ConfigManager

# Load custom config
config_manager = ConfigManager("configs/production.toml")

Configuration Inheritance

Create base configurations and extend them:

# base.toml
[fine_tuner]
base_model_id = "unsloth/Qwen2.5-0.5B-Instruct-bnb-4bit"
epochs = 3
learning_rate = 0.0002

# Override specific values in config.toml
[fine_tuner]
epochs = 5  # Override base value
device_train_batch_size = 8  # Add new value

Configuration Best Practices

1. Version Control

✅ Do: Commit base configuration files ❌ Don't: Commit files with sensitive API keys

# .gitignore
config.local.toml
.env
*.secret.toml

2. Documentation

Always document your configuration choices:

[fine_tuner]
# Using smaller model for development speed
base_model_id = "unsloth/Qwen2.5-0.5B-Instruct-bnb-4bit"

# Optimized for RTX 3080 (10GB VRAM)
device_train_batch_size = 4
grad_accumulation = 4  # Effective batch size: 16

3. Environment-Specific Values

Use environment variables for values that change between environments:

[fine_tuner]
wandb_project_name = "${WANDB_PROJECT:-fine-tuning-dev}"
push_to_hub = "${PUSH_TO_HUB:-false}"

Configuration Sections

Section Purpose Required
[fine_tuner] Model training configuration ✅ Yes
[inferencer] Inference and prediction settings ❌ Optional
[evaluator] Evaluation metrics and settings ❌ Optional

Advanced Configuration

Custom Data Processing

[fine_tuner]
# Custom column mappings
question_column = "input_text"
ground_truth_column = "target_text"
system_prompt_column = "system_message"

# Override system prompt for all examples
system_prompt_override_text = "You are a helpful assistant specializing in..."

Training Optimization

[fine_tuner]
# LoRA configuration
rank = 64              # Higher rank = more parameters
lora_alpha = 32        # Scaling factor
lora_dropout = 0.05    # Dropout for regularization

# Training efficiency
packing = true         # Pack multiple sequences per batch
use_gradient_checkpointing = "unsloth"  # Save memory
dataset_num_proc = 8   # Parallel data processing

Next Steps