# Train a custom Object Detection model

## TL;DR

If you just want to skip explanations and just start training your model, we have cooked a notebook just for you that you can open in Colab.

[Click here to access the notebook.](https://colab.research.google.com/github/PicselliaTeam/picsellia-training-engine/blob/master/PicselliaTrainingQuickStart.ipynb)

## Installation

First we need to install the `picsellia` modules to get started.

```
pip install picsellia
```

And then install one of the following package whether you want to work with Tensorflow 1 or Tensorflow 2.

```python
pip install picsellia_tf1 # Tensorflow 1
pip install picsellia_tf2 # Tensorflow 2
```

{% hint style="info" %}
For the sake of our tutorial, we will use the `picsellia_tf2` package, but the methods are nearly the same with the `picsellia_tf1` package, you can check out the [reference](https://picsellia.gitbook.io/picsellia/references-1/python-training-reference#installation) to learn more about how each module works.
{% endhint %}

Now that we are all set, let's get started !

## Initialize our client

In order for your code to interact with the platform you need to start with those lines.

```python
from picsellia.client import Client
from picsellia_tf2 import pxl_utils
from picsellia_tf2 import pxl_tf

api_token = '4d388e237d10b8a19a93517ffbe7ea32ee7f4787'
project_token = '9c68b4ae-691d-4c3a-9972-8fe49ffb2799'

experiment = Client.Experiment(api_token=api_token, project_token=project_token)
```

Ok, those first lines might need some explanations.

The `Client` is the main Picsell.ia class, it will be used in every tutorials. You can check the reference [here](https://picsellia.gitbook.io/picsellia/references-1/python-sdk-reference-1). The Client is for general use that's why here we initialize the subclass Experiment in order to focus on the experiment part of Picsell.ia.

`pxl_utils` and `pxl_tf` are the only two modules we need to perform training with Tensorflow.

You can find your `api_token` in your profile on the platform, see [this](https://picsellia.gitbook.io/picsellia/getting-started-2/broken-reference).<br>

{% hint style="info" %}
This tutorial assumes that you have created your first project, please refer to [this tutorial](https://picsellia.gitbook.io/picsellia/start-using-picsell.ia#1-what-is-a-project) if it's not the case.
{% endhint %}

You should see a greeting message in your console.

```python
Hi Pierre-Nicolas, welcome back.
```

Now that we are connected to Picsell.ia, let's get down to real business.

## Initialize an experiment

### Checkout an existing experiment

If you have already created an experiment on the platform and chose a base model, you might want to retrieve it to launch your training and not create a new experiment, to do this you can call the following method

```python
exp = experiment.checkout('my-new-model', tree=True, with_file=True)
```

Let's explain the parameters :

* 'my-new-model' is obviously the name of the experiment you created earlier
* `tree`, setting this parameter to True will create the folder structure needed to store and organize all files issued from training (records, checkpoints, config, saved\_model ...)
* `with_file`,  this parameters is set to True to fetch on your machine all the files stored under your experiment (checkpoints, config...)

### Create a new experiment

If you want to log and store everything you create or observe during training, you have to create what we call an experiment.

Check out [this page](https://picsellia.gitbook.io/picsellia/getting-started-2/broken-reference) to learn more about the experiment system.

```python
exp = experiment.create(
    name='my_new_model',
    description='Transfer learning with an efficientdet d0 network',
    source='Pierre-Nicolas/efficientdet-d0-coco17-tpu-32'
    )
```

Now we can see that our experiment has been created and that we have retrieved all the assets from the efficientdet network to start our training :

```python
{
 'id': '3fb04130-1718-4b73-a850-e58dda3d9cfe',
 'date_created': '2020-12-14T18:18:38.010144Z',
 'last_update': '2020-12-14T18:18:38.009873Z',
 'owner': {'username': 'Pierre-Nicolas'},
 'project': {'project_id': '9c68b4ae-691d-4c3a-9972-8fe49ffb2799',
  'project_name': 'project 21'},
 'name': 'my_new_model',
 'description': 'Transfer learning with an efficientdet d0 network',
 'status': 'started',
 'logging': None,
 'files': [
  {'id': 31,
   'date_created': '2020-12-14T18:18:38.017601Z',
   'last_update': '2020-12-14T18:18:38.017435Z',
   'large': False,
   'name': 'config',
   'object_name': 'efficientdet_d0_coco17_tpu-32/checkpoint/pipeline.config'},
  ...
  {'id': 34,
   'date_created': '2020-12-14T18:18:38.052955Z',
   'last_update': '2020-12-14T18:18:38.052806Z',
   'large': False,
   'name': 'checkpoint-index-latest',
   'object_name': 'efficientdet_d0_coco17_tpu-32/checkpoint/ckpt-0.index'}
 ],
 'data': [
   {
    'id': 8,
    'date_created': '2020-12-14T18:18:38.064439Z',
    'last_update': '2020-12-14T18:18:38.064268Z',
    'name': 'labelmap',
    'data': {
     '1': 'person',
     '2': 'bicycle',
     ...
     '89': 'hair drier',
     '90': 'toothbrush'
    }
   }
  ]
}
```

## Prepare for training

Now that we have created our experiment we need to do a few more steps before we can train, i'm sure you've guessed them :

* Download annotations
* Download images
* Perform a train/test split
* Create tfrecords (data placeholder optimized for training)
* Edit the config file (needed to tune our experiment)

Here are the functions that will perform those actions, if you need more information please check the reference [here](https://picsellia.gitbook.io/picsellia/references-1/python-training-reference).

```python
experiment.dl_annotations()
experiment.dl_pictures()
experiment.generate_labelmap()
experiment.log('labelmap', experiment.label_map, 'labelmap', replace=True)
experiment.train_test_split()
```

Now let's fetch the default parameters for our mode, if you want to change some, don't forget to log them back afterward !

```python
parameters = experiment.get_data('parameters')
print(parameters)
{ 
    'steps': 10000,
    'learning_rate': 1e-3',
    'annotation_type': 'rectangle',
    'batch_size': 8
}
parameters['steps'] = 50000
experiment.log('parameters', parameters, 'table', replace=True)
```

```python
pxl_utils.create_record_files(
        dict_annotations=experiment.dict_annotations, 
        train_list=experiment.train_list, 
        train_list_id=experiment.train_list_id, 
        eval_list=experiment.eval_list, 
        eval_list_id=experiment.eval_list_id,
        label_path=experiment.label_path, 
        record_dir=experiment.record_dir, 
        tfExample_generator=pxl_tf.tf_vars_generator, 
        annotation_type=parameters['annotation_type']
        )
```

```python
picsell_utils.edit_config(
        model_selected=experiment.model_selected, 
        config_dir=experiment.config_dir,
        record_dir=experiment.record_dir, 
        label_map_path=experiment.label_path, 
        num_steps=parameters['num_steps'],
        batch_size=parameters['batch_size'],
        learning_rate=parameters['learning_rate'],
        annotation_type=parameters['annotation_type'],
        eval_number = 5,
        incremental_or_transfer=incremental_or_transfer
        )
```

## Train

Now we can call this simple method to run the training

```python
pxl_utils.train(
        ckpt_dir=experiment.checkpoint_dir, 
        config_dir=experiment.config_dir
    )
```

## Evaluate

Call this method to run an evaluation on all our test images

```python
pxl_utils.evaluate(
    experiment.metrics_dir, 
    experiment.config_dir, 
    experiment.checkpoint_dir
    )        
```

## Export the model

Now we can export our model so we can load it to perform inference later on&#x20;

```python
pxl_utils.export_graph(
                       ckpt_dir=experiment.checkpoint_dir, 
                       exported_model_dir=experiment.exported_model_dir, 
                       config_dir=experiment.config_dir
                       )
```

## Run inference

Let's try our model on a few images to check the results

```python
pxl_utils.infer(
     experiment.record_dir, 
     exported_model_dir=experiment.exported_model_dir, 
     label_map_path=experiment.label_path, 
     results_dir=experiment.results_dir, 
     min_score_thresh=min_score_thresh, 
     num_infer=5, 
     from_tfrecords=True, 
     disp=False
     )
```

## Log and store

Nice job ! We have performed a complete training in just a few steps, now we will log our metrics and logs to the platform and store our file assets so we can restore it in our next iteration:

```python
metrics = picsell_utils.tf_events_to_dict(clt.metrics_dir, 'eval')
logs = picsell_utils.tf_events_to_dict(clt.checkpoint_dir, 'train')

clt.store('model-latest')
clt.store('config')
clt.store('checkpoint-data-latest')
clt.store('checkpoint-index-latest')
clt.log('logs', logs)
clt.log('metrics', metrics)

```