Database Guillotine
_{by Nat Cagle}

An ArcGIS python script tool that extracts, splits, or dataloads a dataset
based on custom queries, AOI polygons, or specific feature classes.
Report Bug · Request Feature

Table of Contents

About The Project
- Built With
Installation
- Prerequisites
Roadmap
- Usage
Contact
- License
- Acknowledgments

About The Project

The Database Guillotine is used to extract, split, or dataload TDS datasets. You can split a local dataset or a connected SDE dataset. The tool can create a new GDB and clone the source schema, or dataload the source into a GDB that already has data or an empty GDB. The schemas must match to do this. The data can be extracted based on the full extent of the dataset or with a user created AOI. The AOI must be merged into one polygon (multiparts are allowed). By default, the Extract Scale option has a drop-down list of CTUU values to choose from. The tool will extract all data with a CTUU greater than or equal to the chosen value. All options can be used in combination with each other.

In the Advanced Options section, a custom query can be used to extract data. Choosing this option will cancel out the Extract Scale option above. The Custom Query field has a button to access the Query Builder interface for constructing your query. In order to view field names in the Query Builder, you must first choose a feature class from the Field Name Reference drop-down. The Query Builder references the chosen feature class to make the list of field names shown in the interface. Different feature classes have different fields, but many of them overlap. You can use any feature class to find the CTUU field, but only TransportationGroundCrvs will have the RIN_ROI field. TransportationGroundCrvs is a good option for the most common field names.

The next advanced option is to Extract Specific Feature Classes. Once the dataset input is added, the Feature Class List for this option will populate with all the available feature classes. They are all checked by default. You can uncheck specific feature classes you do not want extracted, such as StructurePnt/StructureSrf. Or you can unselect all of them at once and then check only a few feature classes. Be aware that the output database will still have all the feature classes in the dataset, but if they were not extracted, there won't be any features in them.

(back to top)

Built With

Prerequisites

ArcMap Desktop 10.5+
Python 2.7

Installation

Download the Database Guillotine vX.tbx ArcGIS Toolbox
In ArcMap or ArcCatalog, open the toolbox and double click "Database Guillotine"

(back to top)

Roadmap

(back to top)

Usage

Create new GDB and clone source schema

This will create a new GDB in the chosen folder and clone the schema from the source dataset. This ensures that all the features being extracted match the source they are coming from.

Name for Split GDB

Give your new pet database a cute name!

The name cannot contain spaces and you don't need to add ".gdb" to the end. If you do add .gdb or have spaces in the name, the tool will automatically correct it. Feel free to try all you like; your efforts will be in vain.

Destination folder for split GDB

Choose the folder location where the new GDB will be created.

Dataload into existing data or empty GDB

Instead of creating a new GDB, source features can be dataloaded into a provided GDB with a matching schema. This can replace the ArcCatalog Data Loader tool and its various Cross-Reference databases.

If the provided GDB is empty, only the extracted source features will appear in the GDB when the tool is finished.

If the provided GDB already has existing features, then the extracted source features will be dataloaded together with existing features.

Use Full Extent

This option forgoes using an AOI polygon and just pulls all the data (still following the queries below).

This can be used if you want to pull an entire SDE dataset local for final finishing and delivery.

AOI

This is the AOI that will be used to split the data when extracting. The polygon provided must be merged into one feature. If the AOI you are extracting is not contiguous, still merge the polygons into one multipart feature.

All features that cross this AOI boundary will be split at the edge, retaining their attribution. Any multipart features created as a result of this process are exploded as part of the tool.

Extract Scale

Choose a scale value from the drop-down. The tool will extract data that has a CTUU value greater than or equal to the chosen value.

If you wish to only extract data equal to one specific CTUU value or data that is less than a certain CTUU value, use the Custom Query in Advanced Options.

Extract data using custom query

Advanced Option

This will search the source dataset for any features with attributes that match the provided query. All features that meet the criteria will be extracted.

Field Name Reference

Advanced Option

In order to view field names in the Query Builder, you must first choose a feature class from the Field Name Reference drop-down. The Query Builder references the chosen feature class to make the list of field names shown in the interface.

Different feature classes have different fields, but many of them overlap. You can use any feature class to find the CTUU field, but only TransportationGroundCrv will have the RIN_ROI field. TransportationGroundCrv is a good option for the most common field names. If you want to find the HGT field in the Query Builder, you can select any feature class with that field as a reference, such as UtilityInfrastructurePnt.

Custom Query

Advanced Option

Here you can write a custom SQL query that be used to filter the data being extracted from the source. Choosing this option will cancel out the Extract Scale option above. The Custom Query field has a button to access the Query Builder interface for constructing your query. In order to view the fields in the Query Builder, you must first choose a feature class from the Field Name Reference drop-down. The Query Builder references the chosen feature class to make the list of fields shown in the interface.

Extract Specific Feature Classes

Advanced Option

This allows you to specify any feature classes you would like to include or exclude when extracting data from the source.

Feature Class List

Advanced Option

Once the dataset input is added, this list will populate with all the available feature classes. They are all checked by default.

You can uncheck specific feature classes you do not want extracted, such as StructurePnt/StructureSrf. Or you can unselect all of them at once and then check only a few feature classes.

Note: The output database will still have all the feature classes in the dataset, but the feature classes selected in this list are the only ones that will have data extracted. If a feature class was unchecked from this list, the feature class in the output will be empty.

What is it for? Nobody knows