Versioning Files

Versioning is an important component to reusable, reproducible research. Synapse maintains a version history for every file, making it easy to reference your work from a specific point in time.

When you upload a file to Synapse for the first time, it automatically gets a version of 1. It can be referred to explicitly by its Synapse ID, syn12345678.1. Uploading a new version of a file replaces the existing file in Synapse while preserving the previous version. The Synapse ID will remain but the version will increase, for example, syn12345678.2. It is important to note that, by default, any previous versions of the file are still available. These older versions can be used in provenance relationships, for minting Digital Object Identifiers, or as part of a data release.

Providing the Synapse ID without any versioning information to any of the clients (e.g., syn12345678) will always point to the most recent version of the file. In this way, the most recently updated file is automatically fetched when the version number is omitted.

The easiest way to create a new version of an existing Synapse file is to use the same file name and store it in the same location as the original file (e.g., the same parentId). Synapse will automatically determine that a new version of a file is being stored only if the contents of the file have changed. If the contents have not changed, a new file will not be uploaded and the version number will not increase.

For each file version created, Synapse records the file contents and any associated annotation information. Other metadata about a file (such as the description, name, parent, ACL, and associated wiki) are not recorded as part of the version and will not change between versions.

Uploading a New Version via the Synapse UI

Uploading a new version follows the same steps as uploading a file for the first time. Use the same file name and store it in the same location. It is recommended to add a comment to the new version in order to easily track differences at a glance.

Navigate to the file on Synapse and click File Tools. Select Upload A New Version of File from the dropdown menu and upload or link to your file in the resulting pop-up window.

Once the new version has been uploaded, click File Tools and select Version History. Then select Edit Version Info to add the version comment.

Uploading a New Version Programmatically

Uploading a new version follows the same steps as uploading a file for the first time. Use the same file name and store it in the same location. It is recommended to add a comment to the new version in order to easily track differences at a glance. Version comments are available in Python and R clients, but not in the command line client.

Command line

# Upload a new version of raw_data.txt 
synapse store raw_data.txt --parentId syn123456
#Currently there is no option to add a version comment when uploading via command line. We recommend adding the comment via the web client.

Python

# Upload a new version of raw_data.txt, EXPLICIT UPDATE EXAMPLE
import synapseclient

# fetch the file in Synapse
file_to_update = syn.get('syn2222', downloadFile=False)

# save the local path to the new version of the file
file_to_update.path = '/path/to/new/version/of/raw_data.txt'

# add a version comment
file_to_update.versionComment = 'Added 5 random normally distributed numbers.'

# store the new file
updated_file = syn.store(file_to_update)

# Upload a new version of raw_data.txt, IMPLICIT UPDATE EXAMPLE
# Assuming that there is a file created with: 
syn.store(File('path/to/old/raw_data.txt', parentId='syn123456'))

# To create a new version of that file, make sure you store it with the exact same name
new_file = syn.store(File('path/to/new_version/raw_data.txt',  parentId='syn123456'))

R

# Upload a new version of raw_data.txt, EXPLICIT UPDATE EXAMPLE
library(synapser)

# fetch the file in Synapse, where "syn2222" is the synID of the file in Synapse
file_to_update <- synGet('syn2222', downloadFile=FALSE)

# save the local path to the new version of the file
file_to_update$path <- '/path/to/new/version/of/raw_data.txt'

# add a version comment
file_to_update$versionComment <- 'Added 5 random normally distributed numbers.'

# store the new file
updated_file <- synStore(file_to_update)

# Upload a new version of raw_data.txt, IMPLICIT UPDATE EXAMPLE
# Assuming that there is a file created with: 
synStore(File('path/to/old/raw_data.txt', parentId='syn123456'))

# To create a new version of that file, make sure you store it with the exact same name
new_file <- synStore(File('path/to/new_version/raw_data.txt',  parentId='syn123456'))

Downloading a Version via the Synapse UI

Navigate to where the file is stored in Synapse and click File Tools and Version History to display a list of all file versions. Select the version you would like to download and, once the page has refreshed, select Download Options and Download File.

Downloading a Version Programatically

By default, the file downloaded will always be the most recent version. However, a specific version can be downloaded by accessing the version history or by passing the version parameter.

Command line

# Retrieve the first version of a file from Synapse
synapse get syn56789 -v 1

Python

entity = syn.get("syn56789", version=1)

R

entity <- synGet("syn56789", version=1)

Deleting a Version via the Synapse UI

Deleting a version removes that version number from your version history but does not renumber the remaining versions. For example, if you have three versions of a file and delete the second version, your version history will display version 1 and version 3.

Navigate to the file and select File Tools and then Version History. Find the version you want to delete and click the small “x” on the right side of the corresponding row.

Deleting a Version Programatically

A specific file version can be deleted by accessing the version history or passing the version parameter. Deleting a version removes that version number from your version history but does not renumber the remaining versions. For example, if you have three versions of a file and delete the second version, your version history will display version 1 and version 3.

Command line

synapse delete syn56789 -v 1

Python

entity = syn.delete("syn56789", version=1)

R

# Calling `synDelete` returns NULL
synDelete("syn56789", version = 1)

Updating a File Without Changing Versions

If you are using the Synapse UI to update your files, adding or editing annotations and provenance will not update the file version.

Refer to Annotating Data With Metadata for instructions on adding/editing annotations via the Synapse UI.

If you are using programmatic clients to update your files, any change to a file will automatically update the version, including changes to annotations and provenance. In some rare cases, you may not want to create a new version when you make changes to a file. The main function for storing or updating an entity in the Python and R clients is the store function (see the Python docs or R docs for more information). This function takes an optional forceVersion parameter, whose default value is True. This means that whenever store is called to update an existing entity, the version is increased even if nothing has changed on the entity. If you specifically do not want to change the version when using the store function, include a forceVersion=False parameter.

Updating Annotations Without Changing Versions

Command line

The commands set-annotations and set-provenance will update the metadata without creating a new version.

# Set annotation on a file (syn56789)
synapse set-annotations --idsyn56789 --annotations '{"fileType":"bam", "assay":"RNA-seq"}'

Python

Use the forceVersion=False command with the Python client.

# Get file from Synapse, set downloadFile=False since we are only updating annotations
file = syn.get('syn56789', downloadFile=False)
# Add annotations
file.annotations = {"fileType":"bam", "assay":"RNA-seq"}
# Store the file without creating a new version
file = syn.store(file, forceVersion=False)

R

In the R client, you can use the synSetAnnotations function to modify annotations without creating a new file version.

# Get file from Synapse, set download=False since we are only updating annotations
file <- synGet('syn56789', downloadFile=FALSE)
# Add annotations
annotations <- synSetAnnotations(file, annotations=list(fileType = "bam", assay = "RNA-seq"))

Setting Provenance Without Changing Versions

Refer to Provenance for instructions on adding/editing provenance via the Synapse UI.

Command line

synapse set-provenance -id syn56789 -executed ./path/to/example_code

Python

Because Provenance is tracked by version, set forceVersion=False for minor changes to avoid breaking Provenance.

# Get file from Synapse, set downloadFile=False since we are only updating provenance
file = syn.get('syn56789', downloadFile=False)
# Add provenance
file = syn.setProvenance(file, activity = Activity(used = '/path/to/example_code'))
# Store the file without creating a new version
file = syn.store(file, forceVersion=False)

R

Because Provenance is tracked by version, set forceVersion=False for minor changes to avoid breaking Provenance.

# Get file from Synapse, set download=False since we are only updating annotations
file <- synGet('syn56789', downloadFile=FALSE)
# Add provenance
act <- Activity(name = 'Example Code', used = '/path/to/example_code')
file <- synStore(file, activity=act, forceVersion=FALSE)