What is Spreadsheet Ingest?

The spreadsheet ingest module offers users the ability to batch ingest content into the repository using a spreadsheet provided by the CTDA and does not require files to be compressed into individual zip packages.

Spreadsheet Workflow

Spreadsheet Ingest Permissions

Before attempting to use Spreadsheet Ingest, please contact the CTDA at ctda@uconn.edu to set up the correct permissions to use this module.

The workflow for using the spreadsheet ingest module is split into six sections:

If you have any issues or questions while using the module, please contact ctda@uconn.edu.

1. Connect to UConn VPN

UConn NetID

To complete the following steps, you will need to have a UConn Net ID the correct UConn VPN software installed on your machine. First time users please visit the Configuring UConn VPN guide for instructions on installing UConn's VPN application, PulseSecure. If you do not have a Net ID, please contact ctda@uconn.edu.

  • Open PulseSecure on your computer and connect to the UConn VPN.
  • Enter your UConn Net ID and password.

  • Click the "Connect" button.

  • Once you see the "Connected" status, you can now connect to the CTDA servers via SFTP.

What is SFTP and Why Do We Use It?

A secure file transfer protocol is an application through which you can associate the appropriate files with the objects in the spreadsheet. Windows operating systems have a built-in SFTP.

If you run a different system, you may have to download an application or software. There are several free options including FileZilla. Please check with your IT staff before downloading and installing any software on to your computer.

We need to use an SFTP in the spreadsheet ingest process to give users a secure way to transfer digital objects to CTDA servers in order for the repository to ingest the objects.

2. Connect to CTDA Server via SFTP

  • Open the SFTP client of your choice.

    Check with your IT department before you download and install any applications onto your computer.

  • Connect to the CTDA server using the following credentials:
If your FTP client does not have a Protocol option, type sftp://ctda-private1.lib.uconn.edu into the Host/Address field.
  • Once you have connected to the CTDA server, click on the uploads folder and then locate your institution's folder. Folders are designated by your institution's namespace.

  • Move the digital files you plan to ingest into the repository into your institution's folder on the CTDA server.

    You should not upload your CSV file to the CTDA servers. You will upload your CSV file to the Spreadsheet Ingest page following the directions below.

  • Once your files have been transfer, you are ready to upload your spreadsheet to the repository and start the ingest process.

3. Access a Blank Spreadsheet Template

  • Once you have chosen a template, you need to make a copy to your own Google account.
  • Do this by going to File → Make a Copy

  • Once you have saved a copy, you can fill out the spreadsheet for your project.

If you are undertaking a large digital project and plan on using spreadsheet ingest extensively, you should contact CTDA staff at ctda@uconn.edu to discuss creating a customized template for your institution.

4. Enter Metadata

Ingesting Newspapers or Books?

If you are ingesting newspaper issues and pages using a spreadsheet template, please see the Adding Newspapers Using the Spreadsheet Ingest Module guide for instructions on entering information into specific columns in the spreadsheet.

If you are ingesting books using a spreadsheet template, please see the Adding Books Using the Spreadsheet Ingest Module guide for instructions on entering information into specific columns in the spreadsheet.

5. Download Google Sheet as a CSV

  • Once you have filled out your spreadsheet, you need to download the sheet as a CSV file.
  • In Google Sheets go to File → Download → Comma Separated Values (.csv, current sheet).

  • Your spreadsheet will download as a CSV file and you are ready to use the spreadsheet ingest module in the repository.

6. Upload and Ingest Spreadsheet

Log into https://manage.ctdigitalarchive.org

Go to https://manage.ctdigitalarchive.org/islandora_spreadsheet_ingest.

  1. Choose your CSV file. 

  2. Click "Upload".

  3. Choose either "Ingest Template" or "Ingest Template 2.0" from the Template dropdown list depending on the spreadsheet template you are using.

  4. Select "MODS" as the Output DSID.

  5. Click the Transform output check box.

  6. Enter "DC" into the Transform DSID text box

  7. Select "sites/all/modules/islandora_xml_forms/builder/transforms/mods_to_dc.xml" from the Transform XSLT dropdown

  8. Enter the location of your files on the CTDA server after the prefix home/sftp_uconn/chroot/uploads/. In the example in the image, the files that are ready for upload are located at "40002/Videos". If you are ingesting a newspaper title, newspaper issue, book object or compound object, you do not need to enter anything into this field.

  9. If you are ingesting newspaper, book, or manuscript page objects, you should click the check box. You will see options to "Create PDFs", "Generate OCR", "Generate HOCR", or "Aggregate OCR".

  10. The Namespace is your institution's identifying number. Enter it without the colon. If you are unsure of your institution's namespace, you can look for it here or email ctda@uconn.edu.

  11. Make sure the "Ingest immediately" box is checked.

  12. Click the "Batch Ingest" button.

  • A progress bar will appear displaying the progress of the ingest process.

  • Once the process is complete, you will be brought back to the Spreadsheet Ingest page. If you see the message "Processing complete! The batch set (####) has been processed successfully and has been automatically cleared from the queue" your objects have been ingested successfully. Navigate to the collection to which your objects have been ingested to review.

  • If you see the message "Processing complete." that means there were errors in your spreadsheet that caused objects in your spreadsheet to not be ingested into the repository. Scroll down to the bottom of the page and click "Review Spreadsheet Ingest Batch Queue".

  • On the Batch Queue Page locate the batch you attempted to ingest and click "View set".

  • When viewing the items in the batch, locate the "Message" column. This column will alert you as to why your object could not be ingested. Review your spreadsheet, fix the errors and attempt to run the ingest again.