Opening files Created with Other Programs

When opening comma-separated values files you will be prompted with a dialog where you can specify additional options for how the data are to be imported, such as only importing part of a file. For example, you could import a subset of the lines of data, or skip the heading and footnote text. These options can also be used to import data from files that contain multiple data sections where there are tags to indicate the start and end of a data section.

After you fill in the fields and click OK the New Spreadsheet from CSV file dialog will appear. The options on this dialog are described in Copying and Inserting Columns from the Clipboard, which uses the same dialog window.

Genstat supports the import of SAS value labels for SAS v7-9 files. SAS, unlike other systems, stores value labels as custom formats in a separate catalogue file. The format of this catalogue file is undocumented, therefore, Genstat goes through the ‘front door’ and rather than reading directly from the catalogue file, reads value labels from a new file that you create using SAS itself. This file contains the same information as the catalogue file, but is in a more convenient format for Genstat to read.

You will be prompted for this format file when opening an SAS file. Answer Yes to the query ‘Do you want to specify a PROC FORMAT file for the dataset?‘ if you have created this file for the value labels. If you respond with Yes, it will then prompt for a file containing the value labels created with the PROC FORMAT statement (as below). By default, Genstat will look for a file called sas_fmts.sas7bat in the same directory as the import file. The selected file will be used to transfer your variable labels.

To create a file to include the PROC FORMAT statement for Genstat to import the SAS value labels, you will need to run the following small program in SAS:

where ‘<path to SAS file>’ is the directory that contains your import data file.

This procedure creates a SAS file in the directory ‘<path to SAS file>’ that has the format information for each SAS data file. In this case, the file will have the name sas_fmts.sas7bdat and it will be found in the same directory as the import file.

SAS catalogue files not only support conventional value labels (the one-to-one mapping of a string to a single number), but also the mapping of a range of numeric values to a single string (for example, zip code mapped to state). However, Genstat will only import conventional one-to-one value labels from SAS.

When opening Instat files you will be prompted with a dialog where you can specify additional options for how the data are to be imported. The dialog controls the column names used by Genstat and how the three missing values in an Instat worksheet file are interpreted when reading this file into Genstat. Also you can just import a subset of the columns from the Instat file.

When an SPSS file is imported you will be prompted with a dialog where you can specify how Genstat will interpret the missing values in the SPSS files. SPSS data columns can have up to three special missing value codes associated with them. For example, a 1-5 categorical response may have items 7, 8 and 9, which are special missing value codes meaning, ‘Unprepared to answer question‘, ‘Not applicable‘, and ‘Invalid response’ respectively. This dialog allows you to specify how these special codes are handled.

When opening Matlab 5 files you will be prompted with a dialog where you can select the data structure that you want to be imported from the file. A Matlab 5 workspace file can contain multiple structures. Genstat can only select a single structure from a workspace file at present.

1. In the list of structure in the Matlab workspace file, select one of the structures with the mouse then click OK.
2. To retrieve multiple vector structures you will need to open the Matlab 5 again using Spread | Add | Data File and select the additional columns, repeating for as many columns as you require.

Some Matlab structures cannot be imported: these are marked with an X.

If opening a .dcf file from the File | Open menu you will prompted to select the level, records and items that you require from the CSPro survey file. The Select Records dialog allows you to specify the level within the survey, and the records within that level, then the Select Items dialog allows you to select the items within the selected records, and how you want them read into Genstat.

Select records from CSPro survey file
CSPro (Census and Survey Processing System) is software for the design and collection of surveys produced by the US Bureau of the Census. It is freely available for download from https://www.census.gov. Note: registration is required before downloading.

To import CSPro data two files need to be supplied: CSPro data dictionary (*.dcf) and an associated data file. This dialog appears after the dictionary file is opened and allows you to select the records that you want to import. After the records have been selected a second dialog appears where you can choose the items within the selected records (see next heading Select items from CSPRO survey file below).

This dialog allows you to select the survey level within the CSPro data dictionary file, and then multiple records within the selected survey level. A CSPro data dictionary file normally contains a single survey level, but can have up to three levels. For example if a survey was conducted on villages, households and individuals, then there would be three levels in that survey. Records can be extracted for a single survey level at a time. Within records there are a number of survey items that contain a response to a single question. Some items can be further split into sub-items: for example, the individual ID code may contain sub-items for village and household codes. The first two digits may give the village code, then next two digits the household code, and the final two digits the person code within the household.

Once the records have been selected you can also set a number of options that control how Genstat imports the data. CSPro uses value sets that give labels for the numerical or text items in a item, thus specifying groups. The value sets can have two special codes for missing values and not applicable (NA). Numerical codes are used for missing values and NA (e.g. Age = -9, -8 respectively) and these can be left as is or converted to Genstat missing values *. The value sets are not always 1-1 as in Genstat, which has a single level for each label. In CSPro multiple values can be assigned to the same label using either multiple individual assignments or ranges. For example, for age 13-19 can be assigned the label ‘Teenager‘, and or 0,1,2,3,4 = ‘Preschool child‘. A column can have multiple value sets so that it can be categorized in different ways, so age could be grouped in 5 year (0-4, 5-9, 10-14, …) or 10 year blocks (0-9, 10-19, 20-29, …). The value sets can be used to create factors in Genstat. You can chose which type of value sets to make into factors. By default only 1-1 value sets are converted but you can change the option to make all value sets into factors. Value sets which only define valid ranges and special values are not converted to factors.

Select items from CSPro survey file
To import CSPro data two files need to be supplied: CSPro data dictionary (*.dcf) and an associated data file. When the CSPro data dictionary file is opened you are prompted with a dialog where the records are selected (see previous topic above). Once the records have been selected you must supply an associated data file. This dialog will appear after the associated data file has been opened. It can then be used to select items from the selected records.

Example of importing data from a CSPro survey
This example shows the steps for importing data from a CSPro survey. The questionnaire setup and data for a village survey in Africa are stored in two files: cspro.dcf and cspro.dat. The survey contains two records: person and housing. There is a page of data on each person including information on their age, sex, martial status, education, literacy, work and family. The information on each house includes data on the size, facilities, rental status and occupants. There are a number of value sets for some columns that label the numeric codes for these. Some columns have multiple value sets that group the codes in a number of different ways, for example, age has a number of groupings, one for grouping into 5 year bands, another for grouping into 10 year bands, and one for a ten year grouping ignoring those under 10 years old. The value set information can be displayed on a separate page if required.

1. To import data from the survey open the file cspro.dcf using the File | Open menu. By default, on a Windows PC this file is located in C:\Program Files\GenXXEd\Examples.
2. Once the .dcf has been opened you must select the CSPro data file that is described by this .dcf file.
   Usually this file will have the same name but with the extension .dat. However, if another extension has been used you can use the file open dialog to select the file.
3. After both files have been selected a dialog appears where you can the level and records from the file using the Select Records dialog.

   

4. When the level and records have been selected a second dialog will appear where you can select items within the chosen records.

   

5. Using the default options when selecting records and items will produce the following spreadsheet: