Difference between revisions of "Staging Area Generator"

From IMSMA Wiki
Jump to: navigation, search
(26 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
{{TOC right}}
 
{{TOC right}}
{{Under construction | This page is under construction}}
+
The IMSMA Staging Area Generator is a tool that creates a staging area from an {{IMSMANG}} database. It is thus part of the {{IMSMANG}} tool set, but not included in the {{IMSMANG}} installation package. IMSMA Staging Area Generator thus needs to be installed separately. Please contact your [[Information Management Team | GICHD IM advisor]] in order to get the latest installation package.
The IMSMA Staging Area Generator is a tool that allows to create a staging area from an IMSMA database. It is thus part of the {{IMSMANG}} tool set, but not directly integrated into the {{IMSMANG}} installation package. It thus needs to be installed separately. Please contact your country focal point in order to get the latest installation package.
 
  
{{Note | The Staging Area Generator can only be used with {{IMSMANG}} 6.0 and above}}
+
{{Note | The Staging Area Generator can only be used with {{IMSMANG}} 6.0 and higher.}}
 +
==Basic Principles==__NOEDITSECTION__
 +
The Staging Area Generator takes an {{IMSMANG}} database as input and outputs a staging area database, i.e. a flattened version of the IMSMA database, containing only '''approved''' data. The structure of the staging area and a details on the data it contains can be found on the [[Staging Area Database| IMSMA Staging Area Database]] page.
  
== Installing the IMSMA Staging Area Generator==__NOEDITSECTION__
+
When creating the Staging Area database with the [[Using Staging Area Generator | Staging Area Generator]] is possible to choose:
<ol>
+
* which language to use
<li>To install the staging area generator (SAG) double-click on the '''etl.exe''' file..</li>
+
* which Country structure name to use (main or any of the alternative names)
<li>The graphical installer then starts. On the first screen, choose the language you would like to use for the installation and click '''OK'''.
+
* if PostGIS geodata should be created
[[Image:SAGInstallStep1.png|center|200px]]</li>
+
* which projection to use
<li>Navigate the Installation panels of the graphical installer as described below:
+
* to do data post-processing.
<ol>
 
<li>On the Welcome screen, click '''Next'''.</li>
 
<li>On the Infromation screen, click '''Next'''.</li>
 
<li>We strongly recommend to use the default installation path for the Staging Area Generator. '''Click Next'''.</li>
 
<li>If the target folder (C:\IMSMAETLTool) does not already exist a message will be shown, click '''OK'''. </li>
 
<li>In the packages selection panel, click '''Next'''.</li>
 
<li>Wait for the installation progress bar to finish, then click '''Next'''.</li>
 
<li>When the Setup Shortcuts panel displays, click '''Next'''.</li>
 
<li>When the installation has completed successfully, click '''Done'''.</li>
 
</li>
 
</ol>
 
</ol>
 
After the installation, the following menu entry will be visible in Start &rarr; All Programs &rarr; IMSMA:
 
[[File:Staging_Area_Generator_Icon.png]]<br />
 
  
==Using the IMSMA Staging Area Generator==__NOEDITSECTION__
+
It is easy to [[Scheduling the Staging Area creation | schedule the creation]] of the Staging Area database.
===Basic Principles===__NOEDITSECTION__
 
===Where to use it===__NOEDITSECTION__
 
===The Staging Area Generator Graphical User Interface===__NOEDITSECTION__
 
To start the Staging Area Generator, click on the Staging Area Generator Icon in Start &rarr; All Programs &rarr; IMSMA. This will open the following graphical user interface.
 
[[Image:Staging_Area_Generator_GUI.png|center]]<br />
 
{| class="wikitable" width="1000"
 
|-
 
| align="center" colspan="2" | '''Sections in the Staging Area Generator user interface'''
 
|-
 
| width="200pt" | '''Section'''
 
| width="800pt" | '''Description'''
 
|-
 
| align="left" | '''1 - Source Database'''
 
| align="left" | This section defines the connection of the source database, i.e. the {{IMSMANG}} database from which a staging area should be generated. If the source and the target databases should be on the same server, the default connection parameters can be kept. If the staging area is generated on another server, then the IP address of the source server hosting the {{IMSMANG}} database has to be provided in the '''Host''' field. Remember to update the password in the '''Password''' field if you have changed the default imsma password on your {{IMSMANG}} installation.
 
|-
 
| align="left" | '''2 - Target Database'''
 
| align="left" | This section defined the connection to the target database, i.e. the staging area database to be created. The parameters to be provided are the following:
 
* '''Host:''' since the SAG tool has to be run on the target server, the value for this field should always be '''localhost'''
 
* '''Port:''' port of your target Postgres database server installation - the default is 5432 (which is also the port on which IMSMA is running on the source)
 
* '''Database:''' this is the name of the staging area database to be created. You can specify any name here, using '''lower-case alphanumeric characters and underscores'''. If a database with the provided name '''does not yet exist''', the SAG tool will create a database with that name on the target database server. If a database with the provided name '''already exists''', the content will be '''dropped''' and '''recreated''' by the SAG tool. If, for any reason, the database cannot be created, an error message will be displayed. Contact your country focal point if you are unsure how to solve that.
 
* '''User:''' this is the username of the database user that should be the '''owner''' of the staging area database. The default is imsma - however, this will only work if the staging area is created on the same database server on which IMSMA is running. On another, independent, database, there might not be any user named imsma. In that case, another, existing, user can be provided, or the database superuser named ''postgres'' can be used.
 
* '''Password:''' password of the above user.
 
* '''Postgres User:''' the above defined user that was specified as the owner of the staging area database might not have the permissions to actually ''create'' the new staging area database. Therefore, a user with enough permissions has to be provided here. The default is '''postgres''', which is the superuser having all necessary permissions.
 
* '''Postgres Password:''' password of the above user - in case of doubt, leave the default (''password'').
 
|-
 
| align="left" | '''3 - Post Processing SQL Scripts (optional)'''
 
| align="left" | In this optional field it is possible to provide the full path and name to a '''validated SQL file'''. Whatever is specified inside that file will be executed on the staging area database after its generation. This can be valuable in several use cases, for example:
 
* If a staging area with only a subset of the IMSMA data should be generated (in this case the SQL file would include DELETE statements)
 
* If sensitive data (e.g. victim names and addresses) should not be transferred to the staging area (in this case the SQL file would include UPDATE statements to set all the victim names and addresses to NULL)
 
* If data has to be pre-computed for reporting purposes
 
* If additional database views are necessary, also for reporting and analysis purposes
 
Contact your country focal point for help in developing SQL statements according to specific requirements.
 
|-
 
| align="left" | '''Validate Selections'''
 
| align="left" | After having provided all required connection details (sections 1 and 2), click on '''Validate Selections'''. The following actions will be performed:
 
* A connection to the source database will be attempted
 
* A connection to the target database will be attempted - if a database with the specified name does not yet exist, it will be created
 
* Some data will be read from the source database in order to populate the '''Additional Options''' section.
 
|-
 
| align="left" | '''4 - Additional Options'''
 
| align="left" |
 
* '''Conversion Language:'''
 
* '''Gazetteer Seq No:'''
 
* '''Do Geodata:'''
 
* '''Target SRID:'''
 
|-
 
|}
 
[[Image:Staging_Area_Generator_GUI_options.png|center]]
 
 
 
==Scheduling the Staging Area creation==__NOEDITSECTION__
 
  
 
{{NavBox Business Intelligence}}
 
{{NavBox Business Intelligence}}
 
[[Category:VIE]]
 
[[Category:VIE]]

Revision as of 19:54, 29 September 2015

The IMSMA Staging Area Generator is a tool that creates a staging area from an IMSMANG database. It is thus part of the IMSMANG tool set, but not included in the IMSMANG installation package. IMSMA Staging Area Generator thus needs to be installed separately. Please contact your GICHD IM advisor in order to get the latest installation package.

Note.jpg The Staging Area Generator can only be used with IMSMANG 6.0 and higher.

Basic Principles

The Staging Area Generator takes an IMSMANG database as input and outputs a staging area database, i.e. a flattened version of the IMSMA database, containing only approved data. The structure of the staging area and a details on the data it contains can be found on the IMSMA Staging Area Database page.

When creating the Staging Area database with the Staging Area Generator is possible to choose:

  • which language to use
  • which Country structure name to use (main or any of the alternative names)
  • if PostGIS geodata should be created
  • which projection to use
  • to do data post-processing.

It is easy to schedule the creation of the Staging Area database.