Retrieve and update your assets

Here we will learn how to interact with the data and file assets that we have saved to the platform.

Working with data assets

List all your data assets

If you want to retrieve all the data assets you saved for one experiment, here is how :

from picsellia.client import Client

api_token = '4d388e237d10b8a19a93517ffbe7ea32ee7f4787'
project_token = '9c68b4ae-691d-4c3a-9972-8fe49ffb2799'

experiment = Client.Experiment(
    api_token=api_token,
    project_token=project_token
    )
experiment.checkout('my_new_experiment')
experiment.list_data()

 [
  {
   'id': 72,
    'date_created': '2021-02-09T12:32:18.293746Z',
    'last_update': '2021-02-09T12:32:18.293556Z',
    'name': 'parameters',
    'data': {'steps': 200000,
     'nb_gpu': 1,
     'batch_size': 8,
     'learning_rate': 0.005,
     'annotation_type': 'classification'
     },
    'type': 'table'
   }
  ]

Delete all your data assets

If you think that your experiments needs a bit of cleansing, you can delete all the data assets you saved at once like this :

experiment.delete_all_data()

Retrieve an asset

If you want to retrieve a particular data asset, let's say the parameters of your last training, here is how you can do it :

You will only return what is stored in the data field of the data asset, not all the information about the asset.

experiment.get_data('parameters')

{'steps': 200000,
 'nb_gpu': 1,
 'batch_size': 8,
 'learning_rate': 0.005,
 'annotation_type': 'classification'}

Updating an asset

Now let's say that you want to change the value list of parameters for this training, here is how :

parameters = {
 'steps': 5e6,
 'nb_gpu': 8,
 'batch_size': 64,
 'learning_rate': 0.0055,
 'annotation_type': 'detection'
}
experiment.update_data('parameters', data=parameters)

The method return the updated object

{
 'id': 72,
 'date_created': '2021-02-09T12:32:18.293746Z',
 'last_update': '2021-02-09T12:32:18.293556Z',
 'name': 'parameters',
 'data': {'steps': 5000000.0,
  'nb_gpu': 8,
  'batch_size': 64,
  'learning_rate': 0.0055,
  'annotation_type': 'detection'},
 'type': 'table'
 }

Delete an asset

If you want to completely remove a data asset from all your visualizations, here is how :

experiment.delete_data('parameters')

Working with file assets

List all your file assets

If you want to retrieve all the file assets you saved for one experiment, here is how :

from picsellia.client import Client

api_token = '4d388e237d10b8a19a93517ffbe7ea32ee7f4787'
project_token = '9c68b4ae-691d-4c3a-9972-8fe49ffb2799'

experiment = Client.Experiment(
    api_token=api_token,
    project_token=project_token
    )
experiment.checkout('my_new_experiment')
experiment.list_files()

The method return the list of files

[{'id': 22,
  'date_created': '2021-02-09T12:32:18.022694Z',
  'last_update': '2021-02-09T12:32:18.022465Z',
  'large': False,
  'name': 'config',
  'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/pipeline.config'},
 {'id': 23,
  'date_created': '2021-02-09T12:32:18.068900Z',
  'last_update': '2021-02-09T12:32:18.068723Z',
  'large': True,
  'name': 'model-latest',
  'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/0/saved_model.zip'},
 {'id': 24,
  'date_created': '2021-02-09T12:32:18.112582Z',
  'last_update': '2021-02-09T12:32:18.112410Z',
  'large': True,
  'name': 'checkpoint-data-latest',
  'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/ckpt-0.data-00000-of-00001'},
 {'id': 25,
  'date_created': '2021-02-09T12:32:18.156945Z',
  'last_update': '2021-02-09T12:32:18.156767Z',
  'large': False,
  'name': 'checkpoint-index-latest',
  'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/ckpt-0.index'}]

Delete all your file assets

f you think that your experiments needs a bit of cleansing, you can delete all the file assets you saved at once like this :

experiment.delete_all_files()

Retrieve and download a file asset

If you only need to retrieve informations about a file you can use the following method

experiment.get_file('config')

{'id': 22,
 'date_created': '2021-02-09T12:32:18.022694Z',
 'last_update': '2021-02-09T12:32:18.022465Z',
 'large': False,
 'name': 'config',
 'object_name': '9a141ede-03dc-4e6c-a695-38661d9a97c3/pipeline.config'}

If you need to download the file you can use the following method

experiment.download('config')

If the file size exceeds 5Mb or if you get errors while using download, you might need to set the large parameter to True

experiment.download('config', large=True)

Alternatively, you can download the file to a specified folder, just set the path parameter to the path of the folder (must be an existing folder)

experiment.download('config', path='training')

Update a file asset

The update function is there if you need to perform update on file information such as its name.

experiment.update_file('config', name='new-config')

If you call the store method again, it will automatically erase the old file and replace if with the new one

Delete a file asset

If you want to remove a file asset from one of your experiment, here is how to do it.

from picsellia.client import Client

api_token = '4d388e237d10b8a19a93517ffbe7ea32ee7f4787'
project_token = '9c68b4ae-691d-4c3a-9972-8fe49ffb2799'

experiment = Client.Experiment(
    api_token=api_token,
    project_token=project_token
    name='my_new_experiment'
    )

experiment.delete_file('model-latest')

Be aware that this will also permanently delete the file from our storage.