1. Home
2. MSEKERNEL2D procedure

# MSEKERNEL2D procedure

Estimates the mean square error for a kernel smoothing (M.A. Mugglestone, S.A. Harding, B.Y.Y. Lee, P.J. Diggle & B.S. Rowlingson).

### Option

`PRINT` = string token What to print (`summary`); default `summ`

### Parameters

`Y` = variates Vertical coordinates of each spatial point pattern; no default – this parameter must be set Horizontal coordinates of each spatial point pattern; no default – this parameter must be set Vertical coordinates of each polygon; no default – this parameter must be set Horizontal coordinates of each polygon; no default – this parameter must be set How many values of the kernel width to use; no default – this parameter must be set Maximum values for the kernel width; no default – this parameter must be set Variates to receive the values of the kernel width Variates to receive the estimated mean square error for each value of the kernel width

### Description

This procedure calculates an estimate of the mean square error for a kernel smoothing given a particular kernel width. The method used is that of Berman & Diggle (1989). The data required by the procedure are the coordinates of a spatial point pattern (specified using the parameters `X` and `Y`), the coordinates of a polygon within which smoothing is to be performed (specified using the parameters `XPOLYGON` and `YPOLYGON`), the number of values of the kernel width at which to estimate the mean square error (specified using the parameter `NSTEP`), and the maximum kernel width to use (specified using the parameter `HMAX`). The output of the procedure is a variate containing a sequence of `NSTEP` equally-spaced values of the kernel width parameter from `HMAX/NSTEP` up to `HMAX`, and a corresponding vector containing the mean square error for each kernel width. The values of the kernel width and the corresponding mean square error estimates can be saved using the parameters `HVALUES` and `MSE`.

Printed output is controlled using the `PRINT` option. The default setting of `summary` prints the values of the kernel width and the corresponding mean square error estimates under the headings `HVALUES` and `MSE`.

The output of the procedure may be used to select the optimum kernel width to use with the procedure `PTKERNEL2D`. Note that the estimated mean square errors returned by the procedure are, in fact, scaled estimates. The scaling simplifies the calculations but it can produce negative estimates of mean square errors. The scaling is, however, independent of the kernel width, so that the true mean square error has its minimum at the same kernel width as the scaled version.

Option: `PRINT`.

Parameters: `Y`, `X`, `YPOLYGON`, `XPOLYGON`, `NSTEP`, `HMAX`, `HVALUES`, `MSE`.

### Method

A procedure `PTCHECKXY` is called to check that `X` and `Y` have identical restrictions. A similar check is made on `XPOLYGON` and `YPOLYGON`. The procedure then calculates a sequence of `NSTEP` equally-spaced values for the kernel width, starting at `HMAX/NSTEP` and finishing at `HMAX`. It then calls a procedure `PTPASS` to call a Fortran program to calculate the estimated mean square error associated with each value of the kernel width.

### Action with `RESTRICT`

If `X` and `Y` are restricted, only the subset of values specified by the restriction will be included in the calculations. `XPOLYGON` and `YPOLYGON` may also be restricted, as long as the same restrictions apply to both parameters.

### Reference

Berman, M. & Diggle, P.J. (1989). Estimating weighted integrals of the second-order intensity of a spatial point process. Journal of the Royal Statistical Society, Series B, 51, 81-92.

Procedures: `KERNELDENSITY`, `PTKERNEL2D`, `PTK3D`.

Commands for: Spatial statistics.

### Example

```CAPTION     'MSEKERNEL2D example'; STYLE=meta
VARIATE     hickx,hicky
0.069 0.014 0.049 0.057 0.094 0.015 0.106 0.036 0.130 0.126
0.081 0.079 0.027 0.068 0.027 0.069 0.034 0.081 0.066 0.132
0.040 0.115 0.031 0.108 0.040 0.074 0.041 0.074 0.039 0.156
0.023 0.207 0.098 0.142 0.084 0.176 0.053 0.221 0.035 0.300
0.036 0.310 0.050 0.284 0.027 0.327 0.108 0.307 0.134 0.328
0.143 0.353 0.128 0.379 0.105 0.411 0.129 0.385 0.036 0.358
0.030 0.360 0.028 0.354 0.031 0.376 0.030 0.453 0.068 0.453
0.075 0.490 0.047 0.474 0.023 0.486 0.069 0.481 0.076 0.422
0.082 0.458 0.089 0.444 0.107 0.437 0.146 0.444 0.089 0.473
0.078 0.470 0.084 0.513 0.110 0.516 0.132 0.518 0.122 0.539
0.124 0.555 0.124 0.563 0.103 0.537 0.091 0.555 0.118 0.554
0.027 0.539 0.079 0.562 0.025 0.605 0.026 0.587 0.030 0.571
0.039 0.594 0.047 0.582 0.054 0.580 0.077 0.594 0.070 0.635
0.054 0.617 0.042 0.589 0.069 0.567 0.081 0.620 0.092 0.575
0.102 0.571 0.147 0.569 0.131 0.618 0.098 0.620 0.120 0.608
0.081 0.671 0.102 0.676 0.116 0.680 0.118 0.683 0.094 0.701
0.079 0.648 0.065 0.671 0.047 0.701 0.053 0.709 0.035 0.701
0.030 0.692 0.026 0.684 0.047 0.666 0.064 0.694 0.032 0.711
0.024 0.727 0.027 0.741 0.035 0.725 0.040 0.732 0.045 0.721
0.070 0.712 0.045 0.760 0.032 0.772 0.080 0.721 0.078 0.725
0.092 0.732 0.097 0.720 0.136 0.763 0.127 0.773 0.106 0.747
0.131 0.752 0.109 0.772 0.145 0.787 0.095 0.804 0.129 0.807
0.142 0.844 0.135 0.843 0.100 0.794 0.010 0.795 0.027 0.787
0.028 0.799 0.029 0.814 0.064 0.838 0.009 0.819 0.027 0.877
0.038 0.866 0.037 0.882 0.054 0.851 0.066 0.867 0.070 0.903
0.068 0.912 0.063 0.910 0.035 0.900 0.034 0.899 0.024 0.907
0.001 0.864 0.054 0.865 0.061 0.863 0.077 0.867 0.085 0.855
0.116 0.882 0.120 0.886 0.136 0.863 0.142 0.881 0.127 0.899
0.115 0.905 0.101 0.921 0.137 0.911 0.123 0.893 0.101 0.923
0.123 0.925 0.136 0.978 0.127 0.939 0.115 0.942 0.114 0.950
0.108 0.965 0.105 0.959 0.104 0.975 0.081 0.974 0.078 0.952
0.026 0.939 0.035 0.942 0.071 0.931 0.076 0.953 0.065 0.981
0.064 0.981 0.047 0.749 0.041 0.960 0.031 0.965 0.031 0.963
0.052 0.949 0.207 0.918 0.183 0.908 0.172 0.908 0.175 0.880
0.216 0.892 0.219 0.853 0.160 0.864 0.198 0.852 0.278 0.912
0.245 0.912 0.229 0.896 0.240 0.889 0.246 0.896 0.274 0.890
0.219 0.945 0.244 0.947 0.279 0.979 0.288 0.924 0.234 0.943
0.209 0.922 0.168 0.922 0.163 0.932 0.170 0.763 0.163 0.767
0.154 0.767 0.159 0.756 0.157 0.723 0.179 0.748 0.166 0.724
0.281 0.780 0.258 0.768 0.229 0.815 0.238 0.816 0.249 0.829
0.248 0.835 0.279 0.804 0.265 0.802 0.225 0.792 0.228 0.789
0.153 0.831 0.162 0.846 0.167 0.812 0.176 0.815 0.174 0.831
0.203 0.839 0.211 0.827 0.160 0.784 0.186 0.845 0.155 0.607
0.193 0.568 0.174 0.598 0.166 0.610 0.268 0.592 0.258 0.590
0.260 0.582 0.253 0.703 0.276 0.677 0.289 0.672 0.207 0.700
0.156 0.670 0.162 0.680 0.148 0.661 0.161 0.489 0.209 0.495
0.224 0.477 0.228 0.487 0.222 0.457 0.220 0.445 0.238 0.550
0.247 0.519 0.229 0.522 0.221 0.521 0.254 0.498 0.262 0.524
0.159 0.555 0.212 0.527 0.158 0.527 0.201 0.521 0.193 0.325
0.216 0.325 0.286 0.408 0.289 0.409 0.223 0.402 0.229 0.400
0.211 0.395 0.212 0.255 0.184 0.288 0.198 0.062 0.183 0.0
0.148 0.015 0.185 0.013 0.251 0.066 0.256 0.013 0.271 0.016
0.258 0.019 0.234 0.012 0.242 0.126 0.242 0.136 0.268 0.108
0.265 0.088 0.253 0.104 0.256 0.071 0.398 0.154 0.416 0.167
0.424 0.156 0.413 0.182 0.370 0.251 0.419 0.273 0.385 0.273
0.290 0.293 0.394 0.307 0.404 0.312 0.415 0.306 0.417 0.326
0.372 0.368 0.302 0.008 0.320 0.061 0.294 0.066 0.406 0.008
0.289 0.095 0.339 0.131 0.409 0.408 0.404 0.418 0.335 0.407
0.354 0.422 0.310 0.438 0.332 0.426 0.317 0.465 0.338 0.477
0.394 0.436 0.417 0.450 0.427 0.436 0.430 0.447 0.474 0.477
0.329 0.532 0.330 0.531 0.335 0.508 0.353 0.561 0.323 0.613
0.348 0.620 0.347 0.667 0.409 0.649 0.356 0.709 0.304 0.701
0.331 0.718 0.307 0.716 0.323 0.737 0.331 0.727 0.333 0.746
0.342 0.725 0.347 0.727 0.358 0.725 0.360 0.769 0.342 0.771
0.344 0.761 0.346 0.755 0.302 0.752 0.304 0.769 0.318 0.781
0.333 0.763 0.387 0.722 0.425 0.719 0.370 0.775 0.405 0.847
0.419 0.819 0.408 0.833 0.398 0.832 0.390 0.839 0.367 0.830
0.304 0.785 0.308 0.792 0.314 0.802 0.328 0.787 0.329 0.847
0.312 0.654 0.321 0.865 0.325 0.882 0.323 0.902 0.294 0.909
0.380 0.856 0.389 0.927 0.425 0.973 0.384 0.972 0.366 0.973
0.307 0.985 0.291 0.986 0.412 0.946 0.465 0.897 0.463 0.915
0.440 0.987 0.461 0.987 0.497 0.958 0.438 0.756 0.553 0.760
0.576 0.833 0.475 0.834 0.436 0.841 0.513 0.703 0.538 0.522
0.459 0.486 0.490 0.489 0.498 0.471 0.501 0.435 0.501 0.446
0.444 0.435 0.578 0.501 0.543 0.470 0.556 0.465 0.540 0.452
0.448 0.542 0.459 0.537 0.491 0.522 0.475 0.522 0.459 0.527
0.492 0.561 0.456 0.530 0.438 0.350 0.473 0.332 0.473 0.308
0.556 0.354 0.437 0.421 0.446 0.412 0.508 0.410 0.482 0.403
0.487 0.182 0.439 0.157 0.534 0.254 0.569 0.127 0.495 0.119
0.439 0.104 0.459 0.094 0.693 0.006 0.686 0.008 0.708 0.133
0.657 0.128 0.577 0.105 0.608 0.098 0.619 0.132 0.609 0.126
0.631 0.168 0.621 0.201 0.620 0.203 0.617 0.190 0.708 0.184
0.671 0.218 0.619 0.347 0.659 0.354 0.694 0.376 0.682 0.377
0.601 0.385 0.585 0.404 0.616 0.491 0.615 0.492 0.636 0.496
0.639 0.492 0.682 0.442 0.712 0.458 0.706 0.476 0.696 0.466
0.672 0.476 0.658 0.494 0.714 0.501 0.661 0.525 0.673 0.534
0.660 0.562 0.629 0.516 0.617 0.537 0.604 0.543 0.588 0.547
0.584 0.595 0.609 0.597 0.712 0.584 0.703 0.618 0.667 0.702
0.666 0.715 0.579 0.718 0.641 0.739 0.602 0.753 0.695 0.737
0.686 0.751 0.672 0.762 0.656 0.721 0.659 0.795 0.598 0.839
0.619 0.841 0.584 0.855 0.629 0.910 0.585 0.854 0.654 0.851
0.654 0.897 0.661 0.937 0.683 0.948 0.680 0.987 0.677 0.923
0.663 0.958 0.627 0.952 0.639 0.935 0.640 0.983 0.640 0.983
0.623 0.986 0.608 0.979 0.596 0.989 0.639 0.983 0.767 0.922
0.733 0.902 0.747 0.883 0.779 0.873 0.839 0.913 0.813 0.960
0.848 0.981 0.852 0.979 0.854 0.984 0.858 0.944 0.721 0.978
0.782 0.974 0.776 0.947 0.728 0.924 0.737 0.718 0.838 0.782
0.811 0.769 0.821 0.752 0.831 0.744 0.855 0.844 0.823 0.812
0.820 0.805 0.828 0.813 0.827 0.811 0.819 0.813 0.815 0.819
0.758 0.819 0.768 0.831 0.742 0.648 0.789 0.620 0.793 0.585
0.771 0.597 0.763 0.583 0.748 0.604 0.831 0.650 0.803 0.637
0.842 0.635 0.816 0.591 0.804 0.588 0.798 0.706 0.803 0.700
0.806 0.701 0.811 0.665 0.801 0.680 0.832 0.674 0.744 0.703
0.786 0.711 0.750 0.671 0.769 0.495 0.738 0.471 0.756 0.472
0.764 0.470 0.777 0.443 0.753 0.442 0.748 0.453 0.738 0.446
0.731 0.439 0.802 0.465 0.848 0.479 0.823 0.438 0.804 0.545
0.827 0.544 0.830 0.538 0.842 0.555 0.831 0.509 0.741 0.537
0.782 0.550 0.774 0.510 0.727 0.526 0.841 0.338 0.803 0.413
0.816 0.395 0.746 0.407 0.790 0.393 0.725 0.422 0.784 0.389
0.727 0.201 0.773 0.185 0.832 0.149 0.802 0.155 0.854 0.268
0.832 0.222 0.829 0.223 0.825 0.233 0.846 0.256 0.778 0.279
0.789 0.253 0.729 0.231 0.782 0.014 0.794 0.032 0.823 0.015
0.785 0.078 0.748 0.123 0.950 0.021 0.926 0.052 0.943 0.093
0.951 0.080 0.907 0.115 0.895 0.149 0.908 0.148 0.976 0.181
1.000 0.199 0.991 0.199 0.970 0.211 0.944 0.205 0.972 0.264
0.923 0.278 0.916 0.277 0.997 0.234 0.880 0.227 0.898 0.254
0.866 0.293 0.881 0.289 0.906 0.305 0.896 0.340 0.892 0.343
0.883 0.325 0.874 0.330 0.893 0.315 0.913 0.310 0.929 0.315
0.968 0.285 0.932 0.324 0.924 0.333 0.948 0.341 0.942 0.364
0.949 0.359 0.962 0.370 0.998 0.421 0.987 0.292 0.963 0.323
0.951 0.325 0.946 0.325 0.964 0.424 0.951 0.399 0.881 0.394
0.916 0.396 0.922 0.380 0.927 0.399 0.892 0.417 0.885 0.425
0.872 0.406 0.872 0.427 0.908 0.430 0.921 0.501 0.908 0.479
0.877 0.465 0.972 0.444 0.977 0.445 0.953 0.496 0.977 0.534
0.950 0.557 0.890 0.515 0.939 0.511 0.918 0.544 0.905 0.621
0.897 0.607 0.924 0.591 0.955 0.663 0.969 0.657 0.945 0.692
0.968 0.607 0.982 0.604 0.978 0.628 0.966 0.697 0.882 0.662
0.890 0.665 0.890 0.661 0.889 0.652 0.890 0.647 0.927 0.654
0.946 0.660 0.932 0.701 0.903 0.712 0.902 0.688 0.885 0.696
0.902 0.697 0.895 0.747 0.926 0.775 0.930 0.763 0.905 0.760
0.894 0.774 0.895 0.781 0.896 0.782 0.889 0.749 0.892 0.762
0.952 0.736 0.960 0.722 0.973 0.722 0.996 0.736 0.950 0.760
0.950 0.747 0.938 0.748 0.936 0.771 0.969 0.737 0.953 0.791
0.957 0.805 0.959 0.800 0.996 0.802 0.972 0.837 0.974 0.827
0.944 0.826 0.883 0.789 0.874 0.794 0.919 0.792 0.916 0.828
0.908 0.854 0.900 0.841 0.881 0.855 0.868 0.857 0.921 0.884
0.923 0.921 0.934 0.856 0.963 0.904 0.936 0.922 0.968 0.902
0.984 0.931 0.998 0.975 0.987 0.977 0.992 0.986 0.963 0.985
0.944 0.977 0.930 0.277 0.957 0.939 0.994 0.985 0.883 0.958
0.909 0.963 0.925 0.989 0.879 0.985 0.896 0.961 0.891 0.991
0.858 0.984 0.391 0.402 0.421 0.383 :
VARIATE     xpoly; VALUES=!(0,1,1,0)
&           ypoly; VALUES=!(0,0,1,1)
MSEKERNEL2D Y=hicky; X=hickx; YPOLYGON=ypoly; XPOLYGON=xpoly;\
NSTEP=10; HMAX=0.2
```
Updated on March 7, 2019