pypipytorch95% confidence\u2191 26

PyTorch BERT TypeError: forward() got an unexpected keyword argument 'labels'

Full error message

Training a BERT model using PyTorch transformers (following the tutorial here).

Following statement in the tutorial

loss = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask, labels=b_labels)

leads to

TypeError: forward() got an unexpected keyword argument 'labels'

Here is the full error,

TypeError                                 Traceback (most recent call last)
<ipython-input-53-56aa2f57dcaf> in <module>
     26         optimizer.zero_grad()
     27         # Forward pass
---> 28         loss = model(b_input_ids, token_type_ids=None, attention_mask=b_input_mask, labels=b_labels)
     29         train_loss_set.append(loss.item())
     30         # Backward pass

~/anaconda3/envs/systreviewclassifi/lib/python3.6/site-packages/torch/nn/modules/module.py in __call__(self, *input, **kwargs)
    539             result = self._slow_forward(*input, **kwargs)
    540         else:
--> 541             result = self.forward(*input, **kwargs)
    542         for hook in self._forward_hooks.values():
    543             hook_result = hook(self, input, result)

TypeError: forward() got an unexpected keyword argument 'labels'

I cant seem to figure out what kind of argument the forward() function expects.

There is a similar problem here, but I still do not get what the solution is.

System information:

OS: Ubuntu 16.04 LTS
Python version: 3.6.x
Torch version: 1.3.0
Torch Vision version: 0.4.1
PyTorch transformers version: 1.2.0

Solutionsource: stackoverflow \u2197

As far as I know, the BertModel does not take labels in the forward() function. Check out the forward function parameters. I suspect you are trying to fine-tune the BertModel for sequence classification task and the API provides a class for that which is BertForSequenceClassification. As you can see its forward() function definition: def forward(self, input_ids, attention_mask=None, token_type_ids=None, position_ids=None, head_mask=None, labels=None): Please note, the forward() method returns the followings. Outputs: `Tuple` comprising various elements depending on the configuration (config) and inputs: **loss**: (`optional`, returned when ``labels`` is provided) ``torch.FloatTensor`` of shape ``(1,)``: Classification (or regression if config.num_labels==1) loss. **logits**: ``torch.FloatTensor`` of shape ``(batch_size, config.num_labels)`` Classification (or regression if config.num_labels==1) scores (before SoftMax). **hidden_states**: (`optional`, returned when ``config.output_hidden_states=True``) list of ``torch.FloatTensor`` (one for the output of each layer + the output of the embeddings) of shape ``(batch_size, sequence_length, hidden_size)``: Hidden-states of the model at the output of each layer plus the initial embedding outputs. **attentions**: (`optional`, returned when ``config.output_attentions=True``) list of ``torch.FloatTensor`` (one for each layer) of shape ``(batch_size, num_heads, sequence_length, sequence_length)``: Attentions weights after the attention softmax, used to compute the weighted average in the self-attention heads. Hope this helps!

API access

Get this solution programmatically \u2014 free, no authentication.

curl https://depscope.dev/api/error/277ee8a76ec6415fe315d3ff642fc37e22f1e4629b02de54e41619cd9a36c182

hash \u00b7 277ee8a76ec6415fe315d3ff642fc37e22f1e4629b02de54e41619cd9a36c182

PyTorch BERT TypeError: forward() got an unexpected keyword argument &#39;labels&#39;

PyTorch BERT TypeError: forward() got an unexpected keyword argument 'labels'