{"id":1034,"hash":"13949859b0d055bf90733f87fd2f98928bd24ddf8dbf48079ab4e279de701aa9","pattern":"TypeError in SFTTrainer Initialization: Unexpected Keyword Argument 'tokenizer'","full_message":"Question:\nI am trying to fine-tune the Mistral-7B-Instruct-v0.1-GPTQ model using SFTTrainer from trl. However, when running my script in Google Colab, I encounter the following error:\n\nTypeError: SFTTrainer.__init__() got an unexpected keyword argument 'tokenizer'\n\nHere is the relevant portion of my code:\n\nimport torch\nfrom datasets import load_dataset, Dataset\nfrom peft import LoraConfig, AutoPeftModelForCausalLM, prepare_model_for_kbit_training, get_peft_model\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, GPTQConfig, TrainingArguments\nfrom trl import SFTTrainer, SFTConfig\nimport os\n\n# Load and preprocess dataset\ndata = load_dataset(\"tatsu-lab/alpaca\", split=\"train\")\ndata_df = data.to_pandas()\ndata_df = data_df[:5000]\ndata_df[\"text\"] = data_df[[\"input\", \"instruction\", \"output\"]].apply(\n    lambda x: \"###Human: \" + x[\"instruction\"] + \" \" + x[\"input\"] + \" ###Assistant: \" + x[\"output\"], axis=1\n)\ndata = Dataset.from_pandas(data_df)\n\n# Load tokenizer\ntokenizer = AutoTokenizer.from_pretrained(\"TheBloke/Mistral-7B-Instruct-v0.1-GPTQ\")\ntokenizer.pad_token = tokenizer.eos_token\n\n# Load and prepare model\nquantization_config_loading = GPTQConfig(bits=4, disable_exllama=True, tokenizer=tokenizer)\nmodel = AutoModelForCausalLM.from_pretrained(\n    \"TheBloke/Mistral-7B-Instruct-v0.1-GPTQ\",\n    device_map=\"auto\"\n)\nmodel.config.use_cache = False\nmodel.config.pretraining_tp = 1\nmodel.gradient_checkpointing_enable()\nmodel = prepare_model_for_kbit_training(model)\n\n# LoRA configuration\npeft_config = LoraConfig(\n    r=16, lora_alpha=16, lora_dropout=0.05, bias=\"none\", task_type=\"CAUSAL_LM\", target_modules=[\"q_proj\", \"v_proj\"]\n)\nmodel = get_peft_model(model, peft_config)\n\n# Training configuration\ntraining_arguments = SFTConfig(\n    output_dir=\"mistral-finetuned-alpaca\",\n    per_device_train_batch_size=8,\n    gradient_accumulation_steps=1,\n    optim=\"paged_adamw_32bit\",\n    learning_rate=2e-4,\n    lr_scheduler_type=\"cosine\",\n    save_strategy=\"epoch\",\n    logging_steps=100,\n    num_train_epochs=1,\n    max_steps=250,\n    fp16=True,\n    packing=False,\n    max_seq_length=512,\n    dataset_text_field=\"text\",\n    push_to_hub=True\n)\n\n# Initialize trainer\ntrainer = SFTTrainer(\n    model=model,\n    train_dataset=data,\n    peft_config=peft_config,\n    args=training_arguments,\n    tokenizer=tokenizer,  # This causes the error\n)\n\ntrainer.train()\n\nWhat I Have Tried:\n\nI checked the TRL documentation but couldn't find a tokenizer parameter in the SFTTrainer class.\n\nRemoving tokenizer=tokenizer avoids the error, but then I get a different error about tokenization during training.\n\nI tried passing tokenizer inside training_arguments, but that didn't work either.\n\nQuestion:\nIs SFTTrainer expecting the tokenizer to be handled differently in the latest versions of trl? How should I correctly pass the tokenizer to ensure training works?\n\nI would appreciate any insights!","ecosystem":"pypi","package_name":"google-colaboratory","package_version":null,"solution":"In the 0.12.0 release it is explained that the tokenzier argument is now called the processing_class parameter.\n\nYou should be able to run your code as before by replacing tokenizer with processing_class:\n\ntrainer = SFTTrainer(\n    model=model,\n    train_dataset=data,\n    peft_config=peft_config,\n    args=training_arguments,\n    processing_class=tokenizer,  # This causes the error\n)","confidence":0.95,"source":"stackoverflow","source_url":"https://stackoverflow.com/questions/79546910/typeerror-in-sfttrainer-initialization-unexpected-keyword-argument-tokenizer","votes":11,"created_at":"2026-04-19T04:52:12.295983+00:00","updated_at":"2026-04-19T04:52:12.295983+00:00"}