No Title
[2] Cleveland W S and Devlin S J (1988) Locally-weighted regression: An approach to regression analysis by local fitting, J. Am. Statist. Assoc. 83 596 - 610.
[3] Cohen, P.R. (1985), Heuristic Reasoning About Uncertainty: An Artificial Intelligence Approach (Boston: Pitman Advanced Publishing)
[4] Deaton, A. and Muellbauer, J. (1980), "An Almost Ideal Demand System", American Economic Review 70(3), pp. 312-326.
[5] Gabbay, D.M., C. J. Hogger and J. A. Robinson (eds.) (1992), Handbook of Logic in
[6] Artificial Intelligence and Logic Programming. Vol. 3 (Oxford: Clarendon Press).
[7] Konolige, K (1992), "Autoepistemic Logic" in Gabbay et. al. (1992), pp. 217-295.
[8] Moss, S.(1995), "Control Metaphors In The Modelling Of Economic Learning And Decision-making Behaviour", Computational Economics 8, pp. 283-301.
[9] Moss, S. and Edmonds, B. (1996), "A Knowledge-Based Model Of Context Dependent Attribute Preferences For Fast Moving Consumer Goods", submitted to OMEGA.
The technique of local regression developed by Cleveland and Devlin (1988) is a non-parametric method of fitting a regression surface by multivariate smoothing. The technique is capable of fitting a very wide range of surfaces, by virtue of being nonparametric it is not restricted to any particular class of surface or model. It can be used to model nonlinear relationships between data and has proved to be very successful in many applications.
We present a refinement of the technique that allows the form of the regression surface to be constrained to some extent. This form of the procedure is more appropriate to some applications for example questions of applied economics and marketing.
The original form of the technique is described first. The refinement we propose is then given and the outline of an application concludes this paper.
Local regression
This account follows Cleveland and Devlin (1988) closely. Chambers and Hastie (1992) gives a similar account and describes an implementation of the technique in the statistical package S.
We assume the usual set up for regression, namely that we have observations of the dependent variable yi, i = 1, . . ., n and of a vector of p explanatory variables, xi = (xi1, . . . xip). They are related by the equation
yi = j(xi) + ei
where j is some smooth function and ei is an error term.
We estimate j(x) for a typical value of the vector x by regressing yi on the explanatory variables using points xi near x and weighting these points according to their distance from x. The choice of near vectors xi is determined by a parameter r called the span. Small values of r correspond to taking relatively few points close to x and allow for a greater degree of nonlinearity. As r increases, more points are added to the regression and their weights increase towards 1 until in the limit as r approaches infinity the result is identical to that of a ordinary least squares regression.
In order to describe the process more accurately, let di(x) be the Euclidian distance from xi to the vector x = (x1, . . . xp) and order these distances from smallest to largest. If the span r 1 let q equal rn rounded to the nearest integer and put d(q)(x) equal to the q-th ordered distance. On the other hand, when r > 1, let d(q)(x) be the largest of the distances di(x). The weights attached to the points are calculated using the tricube function T(u; t) given by
T(u; t) = (1 - (u/t)3)3 0 u < t
T(u; t) = 0 t u.
For a fixed value of t, as a function of the first argument u, T(u; t) takes the value 1 when u = 0 and tails off to 0 as u increases to t. Define weight wi(x) for the observation xi by
wi(x) = T(di(x); d(q)(x)).
The approximation f(x) to j(x) is now given by regressing yi on xi with weights wi(x). Cleveland and Devlin (op.cit.) describe two methods of making this approximation. In the first method, local linear approximation, the fitting variables are just the independent variables, while in a locally quadratic approximation the independent variables, their squares and cross-products are included in the regression. The nature of the application should help determine which method is more suitable.
Statistical properties of local regression models can be described in a way that is based on the theory of standard linear regression. The linear regression model, i.e.
y = Xb + e
where X is the matrix of explanatory variables, b the coefficients to be estimated and e the error term, leads to the equation for the fitted values y^
y^ = Gy
where G is the matrix X(XtrX)-1Xtr.
The local regression estimate f(x) can be written in the form
f(x) = Âi=1..n li(x)yi (1)
where the li(x) depend on xk for k = 1, . . . , n and the span r but not on the yi. Therefore, the fitted values y^ are given by the equation
y^ = Ly (2)
where L is the matrix of coefficients (li(xj))ij. The residuals ei = yi - y^i can now be written e = (I - L)y where I is the identity matrix. If we assume the error terms to be independent and normally distributed with variance s2, the variance can be estimated by s2 where
s2 = etr e / trace(I - L)(I - L)tr. (3)
This estimate leads in turn to estimates of the variance of the fitted values and tests of goodness of fit, for example an F test for comparing two models. See Cleveland and Devlin (1988) for further information.
A refinement to local regression
Chambers and Hastie (op.cit.) describe a modification of the procedure described above that they call semi-parametric local regression. The refinement we propose is suggested by their modification.
The weights used to form the local approximations are derived from the Euclidian distance between the vectors of explanatory variables x = (x1 . . . xp). We propose instead that the distance between vectors of a set of supplementary variables v = (vi, . . . vr) is used to determine the weights. The supplementary variables could include all or only some of the explanatory variables xi, it could also include variables quite different from the xi. The case where the supplementary variables are a subset of the explanatory variables gives the semi-parametric models of Chambers and Hastie.
The effect of determining the weights according to a supplementary set of variables can be described relatively easily. Take the case of the semi-parametric models first, i.e. where the supplementary variables are a subset of the explanatory variables. Suppose x1, . . ., xq are the supplementary variables and q < p. When the estimate f(x) of the regression surface is made it is only variables x1, . . ., xq that determine the weights used to form the local approximation. Therefore if two vectors x and x' differed only in the values of the variables xq+1, . . ., xp, i.e. those that play no part in determining the weights, the local approximation would be the same for estimating both f(x) and f(x'). Furthermore, in the case of a local linear model, the difference between the values f(x) and f(x') would be given by the linear local approximation applied to the components xq+1, . . ., xp and x'q+1, . . ., x'p. In this sense the model could be said to be linear in xq+1, . . ., xp for fixed x1, . . ., xq. The coefficients of this linear model depend on the variables x1, . . ., xq. This linearity is the origin of Chambers and Hastie's use of the term `semi-parametric' as opposed to the fully non-parametric nature of the method described above.
In the case when some of the supplementary variables are distinct from the explanatory variables, the interpretation is similar to the semi-parametric models. The model can again be regarded as linear (or quadratic) in the explanatory variables that are not included in the supplementary variables and the coefficients of the linear model depend on the supplementary variables.
The only other relationship between the supplementary and explanatory variables to consider is the case when the explanatory variables form a subset of the supplementary variables. This could be interpreted as a local regression model where all the supplementary variables are included in the regression but those that are not included among the explanatory variables have a coefficient of zero imposed on them.
In all cases the analysis of the statistical properties of the models is unchanged, equations (1) to (3) and the estimates and tests that follow from them hold equally well.
The application to marketing we have investigated involves models of time-series data where the supplementary variables consists of just the time or date stamp of the time-series, i.e. this variable can simply be the number of the observation. The resulting models then amount to time varying linear models. At any fixed point in time the model is linear but this linear model changes over the time period of the data series.
These models are clearly well suited to studying situations when it is very likely that the relationship between variables has changed for example the effect on a product's sales of its price both before and after any significant marketing activity.
Generated with CERN WebMaker