Changes

Jump to: navigation, search

Using Staging Area Generator

789 bytes removed, 21:23, 20 February 2020
no edit summary
==Using the IMSMA Staging Area Generator==__NOEDITSECTION__{{TOC right}}
To start the Staging Area Generator, click on the Staging Area Generator Icon in the Start menu → All Programs → IMSMA. This will open the following graphical user interface.
[[Image:Staging_Area_Generator_GUI.png|center]]<br />
* '''Host:''' since the SAG tool has to be run on the target server, the value for this field should always be '''localhost'''
* '''Port:''' port of your target Postgres database server installation - the default is 5432 (which is also the port on which IMSMA is running on the source)
* '''Database:''' this is the name of the staging area database to be created. You can specify any name here, using '''lower-case alphanumeric characters and underscores'''. If a database with the provided name '''does not yet exist''', the SAG tool will create a database with that name on the target database server. If a database with the provided name '''already exists''', the content will be '''dropped''' and the data will be '''recreated''' by the SAG tool. If, for any reason, the database cannot be created, an error message will be displayed. For example, if the specified target database name contains upper-case letters, the following error message will be displayed:
[[Image:SAG_error_msg_wrong_db_name.png|center|450px]]
Contact your [[Information Management Team| GICHD IM advisor ]] if you are unsure how to solve that.{{Note|Please note that if your IMSMA Staging area database is called something else than '''staging''' and you would like to open the staging mxd file in ArcGIS, you need to rebuild the Data Source links for each layer. more information on connecting ArcGIS to the staging area can be found [[Connecting to IMSMA Staging area from ArcGIS|here]].}}
* '''User:''' this is the username of the database user that should be the '''owner''' of the staging area database. The default is imsma - however, this will only work if the staging area is created on the same database server on which IMSMA is running. On another, independent, database, there might not be any user named imsma. In that case, another, existing, user can be provided, or the database superuser named ''postgres'' can be used.
* '''Password:''' password of the above user.
|-
| align="left" | '''3 - Post Processing SQL Scripts (optional)'''
| align="left" | In this optional field it is possible to provide the full path and name to of a '''validated SQL file'''. Whatever is specified inside that file will be executed on the staging area database after its generation. This can be valuable in several use cases, for example:* If a staging area with only a subset of the IMSMA data should be generated (in this case the SQL file would include DELETE statements)* If sensitive data (e.g. victim names and addresses) should not be transferred to the staging area (in this case the SQL file would include UPDATE statements to set all the victim names and addresses to NULL)* If data has to be pre-computed processed for reporting purposes
* If additional database views are necessary, also for reporting and analysis purposes
For more details and examples, see [[Post Processing SQL Scripts]].Contact your [[Information Management Team| GICHD IM advisor ]] for help in developing SQL statements according to specific requirements.
|-
| align="left" | '''Validate Selections'''
* '''Gazetteer Seq No:''' this option allows to specify the flattening of the country structure into the main object tables, if gazetteer alternative names are used in {{IMSMANG}}. Alternative names in the country structure, if defined, have a sequence number. For example, if for each country structure element two alternative names are defined, the first one has sequence number 0 , the second one sequence number 1. If the standard gazetteer names should be used in the staging area, choose '''None'''. Otherwise, choose the appropriate sequence number.
* '''Do Geodata:''' this is a switch to enable/disable the generation of geodata. The staging area can either be created with non-geodata only, or including all the coordinates (points, polylines and polygons), i.e. the data stored in the GEOPOINT, GEOSPATIALINFO and related tables in {{IMSMANG}}. Refer to the section on geodata below for more details.
* '''Target SRID:''' when generating geodata in the staging area, the data can be re-projected. The Target SRID allows to define the projection. The default, 4326, corresponds to '''WGS 84''' and 3857 corresponds to '''WGS84 Web Mercator'''. For a complete list of available projections, issue the following query on an empty staging area (i.e. the one created after clicking on '''Validate Selections'''):<pre>select *from spatial_ref_sys</pre>
You may also consult http://spatialreference.org/ to find the SRID that you would like to use. PostgreSQL is using the EPSG standard.
You find more information about Staging Area Generator and spatial reference systems '''[[SAG Spatial reference system |-here]]'''.
|}
[[Image:Staging_Area_Generator_GUI_options.png|center]]
A log with all the steps is written in this window as the generation process goes on. This might take some time, depending on the size of the source IMSMA database. The range is from very few minutes up to two hours for very big databases. Once the generation is finished, the newly generated staging area database can be explored with a database query tool such as ''pgAdmin3'' or ''Navicat'', and reporting tools can be connected to it.
==Changing the log level of the Staging Area Generator==__NOEDITSECTION__{{Note|This is only required in case of doubt/error, when more details are needed in order to investigate what goes wrong.}}In order to change the log level of the SAG, for example to write a more detailed output to the log file (etl.log in C:\IMSMAETLTool), the following steps are required:# Open a command line window and go to C:\IMSMAETLTool\bin# Extract the configuration file by typing the following in the command line: <pre> C:\IMSMAETLTool\jdk1.6.0_45\bin\jar xf etl-1.0-jar-with-dependencies.jar mylog4j.properties </pre># Open the resulting file mylog4j.properties (in C:\IMSMAETLTool\bin) in an editor (e.g. Notepad++)# Change the line <pre> log4j.category.com.fgm=INFO </pre> to <pre> log4j.category.com.fgm=DEBUG </pre># Bundle the log file in the jar again by typing the following in the command line: <pre> C:\IMSMAETLTool\jdk1.6.0_45\bin\jar uf etl-1.0-jar-with-dependencies.jar mylog4j.properties </pre>More output is now written to the '''etl.log[[Name_Rules_Staging_Area | page]]''' filecontains information about rules for CDF names and known issues.{{NoteContact your [[Information Management Team |This is only required in case of doubt/error, when more details are needed in order to investigate what goes wrongGICHD IM advisor]] if you need help.}}
{{NavBox Business Intelligence}}
[[Category:VIENAA]]
6,632
edits

Navigation menu