Calculations and manipulation

The directive CALCULATE allows arithmetic calculations on the values of any numeric data structure; logical tests can also be done on numerical and textual values. Functions and operators are available for a very wide range of calculations on matrices and tables. Another general directive is EQUATE, which allows values to be copied from one set of data structures to another; the structures must store values of the same mode (for example, numbers or text), but need not be of the same type.

Structure values can be deleted to save space within Genstat; attributes can also be deleted so that the structure can be redefined, for example as another type. Contents of data structures can be compared, to see if they contain the same distinct items, or whether the distinct values in one structure are a subset of those in another. You can also find all the locations where a number, identifier or string occurs within a data structure.

    CALCULATE performs arithmetic and logical calculations
    DELETE allows values and attributes of data structures to be deleted
    EQUATE copies values between sets of data structures
    SETRELATE compares the sets of values in two data structures
    GETLOCATIONS finds locations of an identifier within a pointer, or a string within a factor or text, or a number within any numerical data structure

There are several general directives for manipulating vectors (variates, factors or texts). Units of vectors can be sorted into systematic order or into random order. Boolean arithmetic can be performed on their contents, or you can form all the ways of partitioning them into subsets. A “restriction” can be associated with a vector, so that subsequent statements operate on only a subset of its units. A default length and labelling can be defined for vectors formed later in the job. Facilities for specific types of vector allow interpolation of values for variates, monotonic regression, calculation of regression quantiles, generation of factor values, and concatenation, editing and searching of text.

    SORT sorts units of vectors into alphabetic or numerical order of an index vector, or forms a factor from a variate or text
    SETCALCULATE performs Boolean set calculations on the contents of vectors and pointers
    SETALLOCATIONS runs through all ways of allocating a set of objects to subsets
    RESTRICT defines a “restriction” on the units of a vector
    UNITS defines default length or labelling for vectors defined subsequently in the job
    INTERPOLATE calculates variates of interpolated values
    FRQUANTILES forms regression quantiles
    MONOTONIC fits an increasing monotonic regression
    GROUPS forms a factor (or grouping variable) from a variate or text, together with the set of distinct values that occur
    CONCATENATE concatenates together lines of text vectors
    EDIT line editor for units of text vectors
    TXBREAK breaks a text structure into individual words
    TXCONSTRUCT forms a text structure by appending or concatenating values of scalars, variates, texts, factors or pointers; allows the case of letters to be changed or values to truncated and reversed
    TXFIND finds a subtext within a text structure
    TXPOSITION locates strings within the lines of a text structure
    TXREPLACE replaces a subtext within a text structure

Another general directive lets you run many algorithms from the Numerical Algorithms Group Library, for example to build mathematical models.

    NAG calls an algorithm from the NAG Library

Other facilities for vectors are provided by the procedures in the Genstat Procedure Library, including:

    APPEND appends a list of vectors of the same type
    FACAMEND permutes the levels and labels of a factor
    FACCOMBINATIONS forms a factor to indicate observations with identical values of a set of variates, texts or factors
    FACDIVIDE represents a factor by factorial combinations of a set of factors
    FACMERGE merges levels of factors
    FACPRODUCT forms a factor with a level for every combination of other factors
    FACSORT sorts the levels of a factor according to an index vector
    FACLEVSTANDARDIZE redefines a list of factors so that they have the same levels or labels
    FACUNIQUE redefines a factor so that its levels and labels are unique
    FBETWEENGROUPVECTORS forms variates and classifying factors containing within-group summaries to use e.g. in a between-group analysis
    FDISTINCTFACTORS checks sets of factors to remove any that define duplicate classifications
    FMFACTORS forms a pointer of factors representing a multiple-response
    FFREERESPONSEFACTOR forms multiple-response factors from free-response data
    FREGULAR expands vectors onto a regular two-dimensional grid
    FRESTRICTEDSET forms vectors with the restricted subset of a list of vectors
    FROWCANONICALMATRIX puts a matrix into row canonical, or reduced row echelon, form
    FSTRING forms a single string from a list of strings in a text
    FTEXT forms a text structure from a variate
    FUNIQUEVALUES redefines a variate or text so that its values are unique
    FVSTRING forms a string listing the identifiers of a set of data structures
    GRANDOM generates pseudo-random numbers from probability distributions
    GRMULTINORMAL generates multivariate normal pseudo-random numbers
    JOIN joins or merges two sets of vectors together, based on classifying keys
    MVFILL replaces missing values in a vector with the previous non-missing value
    ORTHPOLYNOMIAL calculates orthogonal polynomials
    QUANTILE calculates quantiles of the values in a variate
    RANK produces ranks, from the values in a variate, enabling for ties
    SAMPLE samples from a set of units, possibly stratified by factors
    SVSAMPLE constructs stratified random samples
    STACK combines several data sets by “stacking” the corresponding vectors
    STANDARDIZE standardizes columns of a data matrix to have mean 0 and variance 1
    SUBSET forms vectors containing subsets of the values in other vectors
    TXPAD pads strings of a text structure with extra characters so that their lengths are equal
    TXSPLIT splits a text into individual texts, at positions on each line marked by separator character(s)
    UNSTACK splits vectors into individual vectors according to levels of a factor
    VEQUATE equates values across a set of data structures
    VINTERPOLATE performs linear and inverse linear interpolation between variates

There are several procedures for calculating or fitting splines, and for manipulating series of observations of a theoretical curve.

    SPLINE calculates a set of basis functions for M-, B- or I-splines
    LSPLINE calculates design matrices to fit a natural polynomial or trignometric L-spline as a linear mixed model
    NCSPLINE calculates natural cubic spline basis functions (for use e.g. in REML)
    PENSPLINE calculates design matrices to fit a penalized spline as a linear mixed model
    PSPLINE calculates design matrices to fit a P-spline as a linear mixed model
    RADIALSPLINE calculates design matrices to fit a radial-spline surface as a linear mixed model
    TENSORSPLINE calculates design matrices to fit a tensor-spline surface as a linear mixed model
    ALIGNCURVE forms an optimal warping to align an observed series of observations with a standard series
    BASELINE estimates a baseline for a series of numbers whose minimum value is drifting
    PEAKFINDER finds the locations of peaks in an observed series

Directives are available for eigenvalue, QR and singular-value decompositions of matrices, and to form the values of SSPM structures.

    FLRV calculates latent roots and vectors (that is, eigenvalues and eigenvectors)
    QRD calculates QR decompositions of matrices
    SVD calculates singular-value decompositions of matrices
    FSSPM calculates values for SSPM structures (sums of squares and products, means, etc.)

Procedures in the Library for operating on matrices include:

    FCORRELATION forms the correlation matrix for a list of variates
    PARTIALCORRELATIONS calculates partial correlations for a list of variates
    FHADAMARDMATRIX forms Hadamard matrices
    FPROJECTIONMATRIX forms a projection matrix for a set of model terms
    FRTPRODUCTDESIGNMATRIX forms summation, or relationship, matrices for model terms
    FVCOVARIANCE forms the variance-covariance matrix for a list of variates
    GINVERSE calculates the generalized inverse of a matrix
    LINDEPENDENCE finds the linear relations associated with matrix singularities
    MPOWER forms integer powers of a square matrix
    POSSEMIDEFINITE calculates a positive semi-definite approximation of a non-positive semi-definite symmetric matrix
    VMATRIX copies values and row/column labels from a matrix to variates and texts

Tables can be formed containing summaries of values in variates: totals, minimum and maximum values, quantiles, numbers of missing and non-missing values, means and variances. Manipulations of multi-way structures include the ability to add various types of marginal summaries to tables, and to combine “slices” of tables, of matrices or of variates.

    TABULATE forms tables of summaries of the values of a variate
    MARGIN calculates or deletes margins of tables
    COMBINE combines or omits “slices” of tables, matrices or variates

Procedures in the Library for operating on tables include:

    BACKTRANSFORM calculates back-transformed means with approximate standard errors and confidence intervals
    MEDIANTETRAD gives robust identification of multiple outliers in 2-way tables
    MTABULATE tabulates data classified by multiple-response factors
    PERCENT expresses the body of a table as percentages of one of its margins
    SVBOOT bootstraps data from random surveys
    SVCALIBRATE performs generalized calibration of survey data
    SVGLM fits generalized linear models to survey data
    SVREWEIGHT modifies survey weights adjusting to ensure that their overall sum weights remains unchanged
    SVSAMPLE constructs stratified random samples
    SVSTRATIFIED analyses stratified random surveys by expansion or ratio raising
    SVTABULATE tabulates data from random surveys, including multistage surveys and surveys with unequal probabilities of selection
    SVWEIGHT forms survey weights
    TABINSERT inserts the contents of a sub-table into a table
    TABMODE forms summary tables of modes of values
    TABSORT sorts tables so their margins are in ascending or descending order
    TCOMBINE combines several tables into a single table
    T%CONTROL expresses tables as percentages of control cells
    VTABLE forms a variate and set of classifying factors from a table

Directives are available for adding and removing branches of trees, and to assist in the construction and use of trees.

    BASSESS assesses potential splits for regression and classification trees
    BCUT cuts a tree at a defined node, discarding nodes and information below it
    BIDENTIFY identifies specimens using a tree
    BJOIN extends a tree by joining another tree to a terminal node
    BGROW adds new branches to a node of a tree

There are also procedures for displaying and pruning trees. These are provide basic utilities for tree-based analysis, and are used by the existing procedures for classification trees, identification keys and regression trees (BCLASSIFICATION, BKEY and BREGRESSION).

    BCONSTRUCT constructs a tree
    BGRAPH plots a tree
    BPRINT displays a tree
    BPRUNE prunes a tree using minimal cost complexity

Formulae and expressions can be interpreted, revised or constructed automatically from the contents of pointers.

    FARGUMENTS forms lists of arguments involved in an expression
    FCLASSIFICATION forms classification sets for the terms in a formula or breaks a formula up into separate formulae (one for each term)
    REFORMULATE modifies a formula or an expression to operate on a different set of data structures
    SET2FORMULA forms a model formula using structures supplied in a pointer

Values can be assigned to dummies and pointers.

    ASSIGN sets values of dummies and pointers

Aspects of the “environment” of the current job can be modified, such as whether or not Genstat starts output from a statistical analysis at the top of a new page, or whether it should pause during interactive output. New defaults can be set for options and parameters. Details of the environmental settings can be copied into Genstat data structures. Attributes of data structures can also be accessed.

    SET sets details of the “environment” of a Genstat job
    SETOPTION sets or modifies defaults of options of Genstat directives or procedures
    SETPARAMETER sets or modifies defaults of parameters of Genstat directives or procedures
    GET gets details of the “environment” of a Genstat job
    GETATTRIBUTE accesses attributes of data structures
    GETNAME forms the name of a structure according to its IPRINT attribute

There are also various specialist mathematical facilities.

    FPARETOSET forms the Pareto optimal set of non-dominated groups
    GALOIS forms addition and multiplication tables for a Galois finite field
    NCONVERT converts integers between base 10 and other bases
    PERMUTE forms all possible permutations of the integers 1…n
    PRIMEPOWER decomposes a positive integer into its constituent prime powers
Updated on March 5, 2019

Was this article helpful?