Import

Import settings are available for admins and managers and offer functionality to import data directly from CSV files. Lists containing the following fields can be imported currently:

  • Name

  • Alias

  • Description

  • Dates

  • GIS data

  • Types

  • Value types

  • References

  • Reference systems

  • Administrative unit

  • Historical place

  • Origin IDs

  • Place hierarchy

Preparation

Automatic imports cause problems regarding data integrity, so proceed with caution. Fixing such problems can be time consuming, so we strongly advise you to:

  • Make an SQL backups before the import of any data; an existing backup not older than a day is enforced

  • Use the preview (enabled by default) and check if the data looks alright

The import operation is encapsulated in a transaction. So if there is an error in the script, nothing will be imported.

The file:

  • Take a look at the example.csv or example_place_hierarchy.csv

  • Make sure the file extension (.csv) is spelled correctly in lower case e.g. my_data.csv

  • Header names are found in the first row of the table

  • Each following row contains one data set; values are separated by commas

  • Text should be enclosed in double quotes (”), especially if they contain commas

Project

To find imported entities after the import, they have to be associated with a project. If no project is available (or not the right one) you have to create a new one. Name and description of said project can be updated later.

Import class

Only one class can be imported at a time, so you have to choose one of the available classes.

Possible import fields

Column headers can contain the following titles. Other titles won’t be imported and an error message will be displayed

  • id - this field has to be unique per project; if you have same IDs like a person and place with id = 1, you can prefix them in the document e.g. person_1, place_1 before importing. Please use only characters, numbers, underscore (_) or hyphen (-).

  • name - required, an error will be displayed if name; the data will not get imported if a name is missing

  • alias - only available for person, group and place, see Alias

  • description - a description can be provided

  • begin_from - used for dates, see Dates

  • begin_to - used for dates, see Dates

  • end_from - used for dates, see Dates

  • end_to - used for dates, see Dates

  • type_ids - used for linking to a type, see Types

  • value_types - used for linking to a value type, see Value types

  • reference_ids - used for linking data to already existing references in the database, see References

  • origin_reference_ids - used for linking data to already existing references in the database within the same project, see Origin references

  • wkt - only available for places and artifacts, see WKT coordinates

  • reference_system_* - used for linking data to already existing external reference systems in the database, see External reference systems

  • administrative_unit_id - only available for places, ID of existing administrative unit

  • historical_place_id - only available for places, ID of existing historical place

  • parent_id - only available for place, ID of a super unit in a place hierarchy, see Place hierarchy

  • openatlas_parent_id - only available for place, ID of a super unit existing in OpenAtlas, see Place hierarchy

  • openatlas_class - only available for place and only used with parent_id, see Place hierarchy

Alias

Alias can be entered as string. Multiple aliases can be separated by semicolon (;). If an Alias contains a comma (,) please surround the whole string with double quotes(”).

Dates

Dates can be entered in the format YYYY-MM-DD. Fill out the begin_from and end_from field for a known timeframe. For a timespan you can use begin_to and end_to in combination with begin_from and end_from. For more information see: Date.

Keep in mind:

  • if the date format is incorrect, it will be displayed in red and won’t be imported

  • if the required fields are missing data (begin_from and/or end_from), values entered in the other fields (begin_to and end_to) will be lost without further warning

  • There are no advanced checks for dates, so end dates can start before begin dates. Check validity after the importing process; for more information see Data integrity checks

Types

It is possible to link entities to one or multiple Type. This can be useful for example when you use a custom type Case studies you can link all of the imported data to this.

  • Type IDs can be entered in the column type_ids

  • you can enter multiple IDs, separated by a space

  • the ID of a Type can be found at the detail view of the specific Type in your OpenAtlas instance

Value types

It is possible to link entities to one or multiple value Type when importing.

  • a Value Type can be entered via the value_types column

  • type ID and corresponding value are separated by a semicolon (;), e.g. 1234;-13.65

  • for each value Type a value is required

  • multiple value type-value pairs are separated by a space

  • The ID of a value Type can be found at the detail view of a specific value Type in your OpenAtlas instance

References

The imported data can be linked to an already existing Reference.

  • Reference ID and pages are separated by a semicolon (;), e.g. 1234;56-78

  • to link a Reference with multiple page numbers, wrap the whole cell in quotation marks, e.g. “1234;IV, 56-78 542;34-23 66;”

  • to link a Reference without page numbers, just add the ID and a semicolon (;) without further information

  • enter multiple Reference separated by a space, e.g. 1234; 56-78 5678;

  • the ID of each Reference can be found at the detail view of said reference in your OpenAtlas instance

Origin references

The imported data can be linked to an already existing Reference within the same import project and through the origin ID (the ID which was given in the import), e.g. “literature_1”.

  • Reference origin ID and pages are separated by a semicolon (;), e.g. literature_1;56-78

  • to link a Reference with multiple page numbers, wrap the whole cell in quotation marks, e.g. “literature_1;IV, 56-78 book_2;34-23 66;”

  • to link a Reference without page numbers, just add the ID and a semicolon (;) without further information

  • enter multiple Reference separated by a space, e.g. literature_1; 56-78 book_2;

  • the origin ID of each Reference can be found at the detail view of said reference in your OpenAtlas instance if you have Show import information enabled in your Profile.

WKT coordinates

For places and artifact (multi)point, (multi)polygon, and (multi)linestring coordinates or GeometricCollection can be imported. Keep in mind to use the WGS84 geodetic system (EPSG 4326). Coordinates will be imported as WKT. It is only possible to import one geometry for each entry. Since the WKT format uses commas, surround tall coordinates with double quotes (“)

Example:

“LINESTRING (12.458533781141528 41.922205268362234, 12.53062334955289 41.917606998887024, 12.52169797441624 41.888476931243254)”

External reference systems

It is possible to link the imported entity to an existing Reference System. Named the coulmn reference_system_* with the name of the external Reference System appended, e.g. reference_system_wikidata. If spaces occur in the name, please substitute them with underscore (_), e.g. reference_system_getty_aat. Each entry consist of two values, separated by a semicolon (;). The first value is the identifier, e.g. Q54123, the second value is the match type (close_match or exact_match)

Example:

Q54123;close_match

Place hierarchy

Use the parent_id or openatlas_parent_id to generate a Place hierarchy together with Feature, Stratigraphic unit, Artifact, and Human remains. The parent_id has to be an origin id of a row in the current import file. The openatlas_parent_id has to be an existing OpenAtlas entity ID with the correct class. To declare, which entry has a specific class, the openatlas_class column is used. Here the following classes can be entered: Place, Feature, Stratigraphic unit, Artifact, and Human remains. This is case-insensitive. For an example, please see: example_place_hierarchy.csv

For questions about the correct place hierarchy structure, please have a look at archaeological sub units at the OpenAtlas Model.

Import options

  • File - select a file you’ll want to import

  • Preview - if this option is selected, nothing will be imported and you see a preview

  • Check for duplicates - if selected, the chosen class e.g. person will be searched for already existing names. The search is not case-sensitive e.g. “King Arthur” would be found even it is spelled “KiNg ArThUr”. If duplicates are found, a warning is printed but doesn’t stop the import

After the import

When the import went through, a summary which data was imported is shown. You can also browse the projects to see which imported entities are associated with them. If advanced layout is enabled, the detail view of an entity shows from which project the entity was imported, which user did the import and the origin ID value.

It is always a good idea to run :doc:`/admin/data_integrity_checks` after each import.