IMSMA Staging Area

What is the IMSMA Staging Area?

The IMSMA staging area is a flattened and read-only version of the IMSMA^NG operational database. In terms of content, the data is an exact copy of the approved information in the IMSMA^NG database, but the structure is much less complex and therefore easier to query. While in the operational IMSMA^NG database the core object data (e.g. the ID and name of a Land) is stored in one table (e.g. in the hazard table) and descriptive attributes (such as single- and multi-select standard and custom defined fields) are stored in separate tables, in the staging area all this information is available in one single table. Therefore, there is no need to write complex queries to get to all the attributes of an object. Ultimately, reporting and data analysis tools can easily be connected to the staging area.
A staging area can be created out of an IMSMA^NG v 6.0 (and higher) database. It is typically used as a reporting database, via lightMINT, MINT or any other reporting/analysis tool.

Structure of the IMSMA Staging Area

Flattening principles

This section requires basic knowledge of the IMSMANG database table structure.

Where to find which data in the staging area
Type of attribute	Where?	Example
Standard text, numeric, date and time attributes	Stored directly in the main object table, similar to IMSMANG	The HAZARD (Land) attribute hazard_localid is stored in a column also called hazard_localid in the HAZARD table of the staging area.
Standard single-selects	Stored directly in the main object table (whereas in IMSMANG the main object table only contains a GUID reference to the IMSMAENUM table)	The Land attribute status is stored in a column called status_enum in the HAZARD table of the staging area. It contains the translated value of the status base value, e.g. Open or Closed in English or the equivalent values in another language that has been specified at staging area generation.
Standard multi-selects	Stored directly in the main object table as comma-separated list of values AND in a normalized way in the <OBJECT>_STD_MULTISELECT table (in IMSMANG the values for a multi select have to be looked up in the <OBJECT>_HAS_IMSMAENUM and IMSMAENUM tables)	The Land object has a multi-select attribute called Marking (Marking Method). In the staging area, the HAZARD table has an attribute marking_method with a comma-separated list of values, for example 'Official Signs, Local Signs'. Additionally, the table HAZARD_STD_MULTISELECT can be joined with the HAZARD table in order to get to the same values as rows. Reusing the above example, there will be two rows, one with 'Official Signs' and one with 'Local Signs' as values (or the translated values if any translations are available).
CDF text, numeric, date and time attributes and CDF single-selects	Stored directly in the main object table (as opposed to IMSMANG where a lookup has to be made through the <OBJECT>_HAS_CDFVALUE, CDFVALUE and CUSTOMDEFINEDFIELD tables)	Each CDF will be turned into a column in the staging area. For example, a CDF called My Hazard CDF defined on Land will result in a column named my_hazard_cdf in the HAZARD table of the staging area. Note that non-standard characters in CDF names, such as blanks, are replaced by underscores in the column names
CDF multi-selects	Stored directly in the main object table as comma-separated list of values AND in a normalized way in the <OBJECT>_CDF_MULTISELECT table (in IMSMANG the values for a multi select have to be looked up in the <OBJECT>_HAS_CDFVALUE, CDFVALUE and CUSTOMDEFINEDFIELD tables)	Let's assume that the Land object has a multi-select CDF attribute called My Land Multi-Select. In the staging area, this will result in a column in the HAZARD table called my_land_multi_select with a comma-separated list of values, for example 'Value1, Value2'. Additionally, the table HAZARD_CDF_MULTISELECT can be joined with the HAZARD table in order to get to the same values as rows. Reusing the above example, there will be two rows, one with 'Value1' and one with 'Value2' as values.
Location data	The guid, localid and name of the location an object is assigned to are directly stored in the main object table.	The HAZARD table in the staging area has the columns location_guid, location_localid and location_name with the values of the location that each object is assigned to.
Organisation data	The guid, localid and name of the organisation defined on an object are directly stored in the main object table.	The HAZARD table in the staging area has the columns org_guid, org_localid and org_name with the values of the organisation defined for each object. If CDFs of type organisation have been created in IMSMA, these will also be available in the staging area.
Place data	The localid and name of the place linked to an object are directly stored in the main object table.	The HAZARD table in the staging are has the columns ammunition_storage_localid and ammunition_storage_name that refer to a place object.
Classification data (Country structure, Assistance Classification, Cause Classification and Needs Assessment Classification)	The guid, localid and name of classifications (country structure (gazetteer), assistance classification, cause classification and needs assessment classification) associated to an object are directly stored in the main object table. Since a classification can have several levels, there is a placeholder for each level, up to the maximum number of levels.	In the HAZARD table in the staging area, there are the following columns regarding the country structure classification (gazetteer): gazetteer_guid, gazetteer_level1_localid, gazetteer_level1_name, gazetteer_level2_localid, gazetteer_level2_name, ..., gazetteer_level7_localid, gazetteer_level7_name. There are seven placeholders because the country structure in IMSMANG can have up to seven levels.

Database model

A simplified staging area database model for the Land (Hazard) object is depicted below. For all the other main objects, the model looks very similar.

Special table

Table name

Description

ETLHISTORY

This table is the only one that does never get dropped from the staging area, but persists as the database is re-generated again and again (see Staging Area Generator). I contains information about all the staging area generations on a specific database (i.e. for example when a staging area database with the same name is generated regularly, e.g. once per day or week). It has two columns: etlruntime: this is the timestamp of the start of the generation runtime: this is the time in minutes that the generation took Having x rows in this table means that the database has been generated x times, and allows to keep a history of the individual runs. It also allows to determine how old the information in the staging area is, by looking at latest timestamp.

Point, Polyline and Polygon tables

If the option of generating geodata has been chosen on the Staging Area Generator interface, then three related tables with geodata are generated for every main object/table (ACCIDENT, VICTIM, QM, HAZARD (Land), etc.):

• <OBJECT>_POINT: contains all the data related to points defined on the object, with the following main information:
  • point_type: e.g. Benchmark, Landmark, Turning Point (all points making up a polygon are also recorded in this table)
  • lat: latitude
  • lon: longitude
  • coordinate_reference_system: e.g. "WGS 1984"
  • coordinate_format: e.g. "Decimal Degrees"
  • shape: PostGIS-representation of the shape that can be accessed with a PostGIS query
• <OBJECT>_POLYLINE: contains all the data related to polylines defined on the object. The individual points making up the polyline are stored in the <OBJECT>_POINT table, but the polyline object is additionally stored in the <OBJECT>_POLYLINE table. The main column is shape, a PostGIS-representation of the shape that can be accessed with a PostGIS query.
• <OBJECT>_POLYGON: similar to <OBJECT>_POLYLINE, but for polygon objects.

Views

A series of database views are pre-defined on the staging area, three for each object: <OBJECT>_POINT_VIEW, <OBJECT>_POLYGON_VIEW and <OBJECT>_POLYLINE_VIEW - where <OBJECT> is HAZARD, GAZETTEER, MRE, etc. - each object that potentially has associated geodata. Each view is defined as an INNER JOIN between the <OBJECT> table, e.g. HAZARD, and each of the three geodata tables. The inner join means that data from e.g. the HAZARD table that has no polygon associated will not be found in the HAZARD_POLYGON_VIEW. Depending on the use case, either the tables or the views can be accessed.