Optimal Kiefer Ordering of Simplex Designs for Third-Degree Mixture Kronecker-Models with three ingredients.

The research was financed by the Deutscher Akademischer Austausch Dienst (DAAD), Abstract This paper investigates the Kiefer optimality in the third-degree Kronecker model for mixture experiments. For mixture models on the simplex, a better design is obtained, by matrix majorization that yields a larger moment matrix due to increase of symmetry and Loewner ordering. The two criteria together constitute the Kiefer design ordering and any such criteria single out one or a few designs that are Kiefer optimal. For the third-degree mixture models with three ingredients, an exchangeable moment matrix was constructed by use of Kronecker product algebra. These moment matrices are symmetrical, balanced, invariant and have homogenous regression entries which are good and have desirable properties for an optimal design. Then, the necessary and sufficient conditions for two exchangeable third-degree K-moment matrices to be comparable in the Loewner matrix ordering were set up. The weights obtained from the original design were used in the construction of the weighted centroid designs. Based on the results obtained, it was shown that the set of the weighted centroid designs constitutes a minimal complete class designs for the Kiefer design ordering and that any design that is not weighted centroid design can be improved upon by convex combination of an appropriate elementary designs. Key words: Kiefer optimality, Kronecker product, weighted centroid designs, simplex centroid design . DOI : 10.7176/MTM/9-1-06


Introduction
The description of the original mixture problem is when two or more ingredients are mixed together to form a product. This product has desirable properties that are of interest to the manufacturers. It is assumed that these properties are functionally related to the product composition and that by varying the composition through the changing of ingredients proportions, the properties of the product will also vary. In the general mixture problem, the measured response is assumed to depend only on the relative proportions of the ingredients present in the mixtures and not on the amount of the mixture Cornell (1990). The study of functional relationship between the measured property (response) and the controllable variables is to determine the best combination of ingredients that yield the desired product. In this basic example of Cake formulations using baking powder, shortening, flour, sugar, eggs and water, the experimenter is looking for fluffiness of the cake, such that fluffiness is related to the ingredient proportions. Similarly, in building construction concrete formed by mixing sand, water, and one or more types of cement building, then the desired property is the hardness or compression strength of the concrete, where the hardness is a function of the percentages of cement, sand, and water in the mix. Cornell (1990) lists numerous examples and provides a thorough discussion of both theory and in practice. Therefore, a mixture experiment involves varying the proportions of two or more ingredients, called components of the mixture, and studying the changes that occur in the measured properties (responses) of the resulting end products. Clearly, if we let q represent the number of ingredients (or constituents) in the system under study and if we represent the proportion of the ith constituent in the mixture by ,then;  0, = 1,2, … , and ∑ = 1+ =1 2+ 2+ … + = 1.0 (2) t ,shown in Eqs.
(1) and (2) the geometric description of the factor space containing the q components consists of all points on or inside the boundaries (vertices, edges, faces, etc.) of a regular (ql)-dimensional simplex. For q = 2 components, the simplex factor space is a straight line. With three components q = 3, the simplex factor space is an equilateral triangle, and for q = 4 the simplex is a tetrahedron.

Mixture Experiments
, that is the factors represent relative proportions of m ingredients blended in a mixture (Cheruiyot et al, 2017). The experimental conditions are points in the probability simplex, which constitute the independent and controlled variables (factors). Replication under identical experimental conditions or responses from distinct experimental conditions are assumed to be of equal (unknown) variance  2 and uncorrelated. The functional relationship between dependent and independent variable within the range of interest is represented by a Taylor polynomial of low degree, d. There are three types of mixture design; simplex-lattice design, simplex-centroid design and simplex axial design. When the mixture components are subject to the constraint that they must sum to one, then standard mixture designs for fitting standard models used are Simplex-Lattice designs and the Simplex-Centroid designs. A simplex design is a mixture design in which the design points are arranged in a uniform way known as lattice. The word, lattice means an array of points and is used in reference to specific Taylor polynomial equation. The simplex centroid design is constructed to form a triangle with data points located at each corner, at the three midpoints on each side and as well as the point located in the centre(centroid). In the simplex lattice design, the points are located on the vertices and mid-edges of an equilateral triangle only, gives more information about response surface behaviour for binary blends, while the points that are located within(inside) the triangle in simplex centroid design and axial design, more uniform distribution is in the interior of the triangle Pukelsheim, 1998a, 1999).

Simplex centroid Designs
The simplex is defined in geometrical terms as a regular figure, where all of its angles are congruent and all of its sides are congruent, such as equilateral (3-sided), tetrahedron (4-triangular faces), and other polygons with triangular faces. In the simplex centroid design, the points are located on the vertices, mid-edges and in the centre (centroid) a triangle .Generally, in a q-component simplex-centroid design, the number of distinct points is 2 − 1.These points correspond to q permutations of (1,0,0, … ,0 ) or q single-component blends, the ( 2 ) permutations of ( 1 2 , 1 2 , 0,0, … ,0 ) or all binary mixtures, the ( 3 )| permutations of ( 1 3 , 1 3 , 1 3 , 0,0, … ,0 ) ,…and so on, with finally the overall centroid point ( 1 , 1 , … , 1 ) or q-nary mixture. In other words, the design consists of every (nonempty) subset of the q components, but only with mixtures in which the components that are present appear in equal proportions. Such mixtures are located at the centroid of the (q-l)-dimensional simplex and at the centroids of all the lower dimensional simplices contained within the (q-l)-dimensional simplex. At the points of the simplex-centroid design, data on the response are collected and a polynomial is fitted that has the same number of terms (or parameters) to be estimated as there are points in the associated design (Muriungi et al, 2017). For example, a case where q = 3 component system and the factor space for all blends is an equilateral triangle, then each component assumes the proportions = 0, 1 2 and 1 for i = 1, 2, 3 (3) Setting d = 2 for the proportions in equation (1), that is the second-degree model is used to represent the response surface over the triangle, then   The three points which are defined as (1, 0, 0) or 1 = 1, 2 = 3 = 0; (0,1,0) or 1 = 3 = 0, 2 = 1 and (0,0,1) or 1 = 2 = 0, 3 = 1  (Cornell, 1990(Cornell, , 2002 The points (  (4), then if the second-degree model is used for a three-component system, we have the expected responses and the polynomial equation of the form;  = 1 1 + 2 2 + 3 3 + 12 1 2 + 13 1 13 + 23 2 3 + 123 1 2 3 which is a polynomial fitted that has the same number of terms (or parameters to be estimated) as there are points in the associated design where 1, , 2 , 3 , 12 , 13 , 23 , 123 unknown parameters.

Methodology
The Kronecker product has been applied in this study to derive the exchangeable moment matrices since Kiefer design ordering does not depend on the coordinate system that is used to represent the regression function, though both Kronecker and the Scheffe' are based on the same space of regression polynomials, but differ in their choice of representing this space. (Draper and Pukelsheim, 1998) and (Prescott, et. Al, 2002) put forward several advantages of the Kronecker model such as homogeneity of regression terms, attractive symmetry, compact notation, great transparency, and invariance properties. We refer to the corresponding expressions as K-models or K-polynomials. In particular, polynomial regression model for mixture experiments as suggested Pukelsheim, 1998a, 1999) in the first and second-degree Kronecker mixture models in which they obtained the results for Kiefer design ordering of mixture experimental design were reviewed.
For a linear model with regression function f (t), the statistical properties of a design  are captured by its moment and its regression function is given by for first, second-and third-degree Kronecker model respectively.

Kronecker product
The Kronecker product, denoted by , is an operation on two matrices of arbitrary size resulting in a block matrix. The Kronecker product should not be confused with the usual matrix multiplication which is an entirely different operation. For a m k  matrix A and an n l  matrix B, their Kronecker product The Kronecker product approach bases second-degree polynomial regression in m variables such that ) , , , on the matrix of all cross products is given by, rather than reducing them to the Box-Hunter minimal set of monomials ). , , , The benefits are that distinct terms are repeated appropriately, according to the number of times they can arise, so that transformational rules with a conformable matrix R become simple )( ( and that the approach extends to third-degree polynomial regression. However, the arrangement of triple products k j i t t t in a set of "layered" matrices appears rather awkward. This is where Kronecker products prove useful; they achieve the same goal with a more pleasing algebra. The idea underlying the use of Kronecker products is familiar from elementary statistics, that is the Kronecker product of a vector  s ℝ m and another vector  t ℝ n then simply is a special case, One of the key property of Kronecker product is the product rule . This greatly facilitates our calculations when we now apply Kronecker products to response surface models of third-degree Kmodel. The first-degree K-model which was proposed by Draper, N. R., Pukelsheim, F, (1998)

The first-degree K-model
If the linear model has regression function t t f = ) ( , which is an identity matrix, the statistical properties of a design  are captured by its moment matrix, The first-degree moment matrix of an exchangeable design  is the with identical on-diagonal entries 2  , the pure second moments and identical off-diagonal entries 11  , the mixed second moments. Furthermore, the simplex restriction entails that is, the entries of any first-degree moment matrix sum to one, for every design on the simplex. For the firstdegree model on the simplex, the regression function is the identity whence the groups  and perm (m) coincide, Therefore the support points of  must be among the vertices i e . Because of exchangeability the design  assigns constant weight m / 1 to each vertex, whence 1   = .Now we view matrix majorization and Loewner ordering together, to obtain the main result on the Kiefer design ordering in first-degree models.

Theorem 1
Among all designs on the simplex  , the unique Kiefer optimal design for a first-degree model is the vertex

Proof:
Let  be an arbitrary design on the simplex T. Lemma 1 yield . The convex combinations of the vertex points design 1  and of the overall centroid design m  exhaust all possible exchangeable first-degree moment matrices.

The second-degree K-model
The second-degree K-model which was proposed by Draper and Pukelsheim (1998b) is of the form An arbitrary design  has second-degree K-moment matrix The K-regression function chosen is We call a design with this invariance property an exchangeable design.

(a) Two factors
For the second-degree K-model with two-ingredient, let  be an arbitrary exchangeable design on ,then, the second-degree K-moment matrix is of the form; The simplex restriction has the effect on moment matrix, that is the entries of any second-degree K-moment matrix sum to one; and its simplex restrictions entail:

Proof:
For the direct part we assume This means .For the converse part note that for two ingredients, equality of second order moments implies equality of third order moments. The fourth order moment differences then are, using In terms of matrices this means 0 E 1 1 1 1 Again the be vertex points design 1  and the overall centroid design 2  play a special role . Hence the weighted centroid design  is well defined. The following theorem joins the partial steps together to obtain the main result on the Kiefer ordering, that the mixtures of the vertex points designs 1  and of the overall centroid design 2  form a minimal complete class.

Theorem 2
In the two-ingredient second-degree model, the set of weighted centroid designs , constitutes a minimal complete class of designs for the Kiefer ordering.

Proof:
Completeness of C means that for every design  not in C there is a member  in C that is Kiefer better than . That is, we must show that  is more informative than , , but that the two are not Kiefer equivalent, .From the above section and with the weights from Lemma 3, the weighted centroid . The implication of the above is that any design which does not consist of a mixture of elementary centroid designs can be improved upon, in terms of symmetry and Loewner ordering, by using an appropriate combination of elementary centroid designs.

(b) Three factors
The second-degree K-moment matrix with three-ingredients is of the form; With an additional moment of mixed order four given by, The simplex restriction has the effect on moment matrix, that is the entries of any second-degree K-moment matrix sum to one; And its simplex restriction entails;

The Third-degree K-model
The third-degree K-model, which was proposed by (Korir, 2008), is of the form;  be an arbitrary exchangeable design on , the third-degree K-moment matrix for two-ingredient is of the form; The simplex restriction has the effect on moment matrix, that is the entries of any K-moment matrix sum to one (Korir, et al, 2009) is called a weighted centroid design. In order to find an appropriate set of weights, we equate selected moments of order lower than four: When the lower order moments are expressed using fourth order moments, these weights are seen to be the ones given in the following Lemma.

Lemma 4 Let
 be an exchangeable designs on the simplex  , with fourth order moments 211 22 31 4 , , , constitutes a minimal complete class of designs for the Kiefer ordering.

Proof:
The Completeness part is as established just as in Theorem 2. For minimal completeness, we remove a weighted centroid  from C and assume that . By Lemma 2.7 (Korir,2008), the two designs share the same lower order moments. The latter determine the weights uniquely, contradicting the assumption that  and  are distinct. Hence the class C is minimal complete. suggested and analysed canonical model forms when the regression function for the expected response is a polynomial of degree one, two, or three. We refer to these as the S-polynomial or S-models. In this paper, the alternative representation of mixture models is used to investigate the third-degree mixture models with three ingredients. This version is based on the Kronecker product algebra of vectors which was introduced by Draper and Pukelsheim (1998Pukelsheim ( , 1999. The Kronecker algebra gives rise to homogeneous model function and moment matrices. We refer to the corresponding expressions as K-models or K-polynomials. In the third-degree mixture model, whereby the S-polynomial and the expected response takes the form;

Design problem
and when the regression function is the homogeneous third-degree K-polynomial, the expected response takes the form (Korir, 2008), (Gregory et al, 2014).

Exchangeability in Third-degree K-model
Given an arbitrary design , we obtain an exchangeable design (permutation invariant)  by averaging over the permutation group, Otherwise the average  is an improvement over  , in that it exhibits more symmetry and balancedness. In terms of matrix majorization, the moment matrix of the average design  is majorized by the moment matrix of ,such that ; (Korir,B.C,2008).
This Therefore, a third-degree K-moment matrix is said to be permutationally invariant when then, we speak of an exchangeable thirddegree K-moment matrix. In a third-degree mixture model, the moment matrix is the form; and has all entries homogeneous of degree six and the simplex restriction has an immediate effect on these moment matrices, as follows That is, all the entries of any third-degree K-moment matrix sum to one; for every design on the simplex.

.2 Kiefer Design Ordering
The optimality properties of designs are determined by their moment matrices (Pukelsheim 1993, chapter 5). We compute optimal design for the polynomial fit model, the third degree Kronecker model. This involves searching for the optimum in a set of competing exchangeable moment matrices (Gregory et al, 2014). The Kiefer partial ordering is a two-stage ordering, reflecting an increase in symmetry by matrix majorization and a subsequent enlargement in the Loewner ordering (Pukelsheim, 2006). In view of the initial symmetrization step, it suffices to search for improvement in the Loewner ordering sense, among exchangeable moment matrices only. First, we obtain the exchangeable moment matrices, then find the necessary and sufficient conditions for two exchangeable third-degree K-moment matrices to be comparable in the Loewner matrix ordering. The comparison of moment matrix inequalities reduces to the comparison of individual moment inequalities which is part of the condition. In terms of matrix majorization relative to the congruence action that is induced on the moment matrices by  (Kennedy, et al, 2015).
Further, the weights were derived from the initial design and these are assigned to the points of support in experimental domain  , these are points on or inside the boundaries (vertices, edges, faces, Centres) of a regular dimensional simplex. These weights were used to obtain the weighted centroid designs in which a convex combination of the elementary centroid designs give rise to the set of weighted centroid designs. Pukelsheim (1993) gives a review of the general design environment. Klein (2002) showed that the class of weighted centroid designs is essentially complete class for m ≥2 for the Kiefer ordering design. As a consequence, the search for optimal designs may be restricted to weighted centroid designs for most criteria.

.Third degree K-model with Three Factors
In the third-degree model, with three-ingredients proposed by Korir, B.C.(2008), an exchangeable moment matrix on design  is of the form where A, B, C, D, F, and G are 9 9 block matrices as follows, That is, the entries of any third-degree K-moment matrix sum to one,  The set of moments of order six determines all lower order moments. For instance, the pure fifth moments expand to order six by 51 6 3 2 1 In this way we get the following relations: be the vector of moments up to order five as given in lemma 3.1 (Korir,.2008), then the sixth order moment differences are as follows; Lemma 5 Let   and be two exchangeable designs on the simplex  . Then we have
There are three elementary centroid designs: 1  is supported on the vertices, 2  on the edge midpoints and 3  the overall centroid point The moments of order six of these designs are respectively (   1  321  1  222  1  411  1  33  1  42  1 (   3  321  3  222  3  411   3  33  3  42  3 In order to find an appropriate set of weights for  In the three-ingredient third-degree model, the set of weighted centroid designs (Korir, 2008)      + + = ) , , ( :

Proof:
The Completeness of C means that for every design  not in C there is a member  in C that is Kiefer better than . That is, we must show that  is more informative than , means that .The implication of the above proof is that any design which does not consist of a mixture of elementary centroid designs can be improved upon, in terms of symmetry and Loewner ordering by using an appropriate combination of elementary centroid designs.

Conclusion
This study investigated the Kiefer design ordering in the third-degree Kronecker model for mixture experiments. For mixture models on the simplex, the improvement of a given design is obtained, by increase of symmetry that yields a larger moment matrix under the usual Loewner ordering. The two criteria together constitute the Kiefer design ordering. For the third-degree mixture models, three ingredients, an exchangeable moment matrix was obtained for each factor-case, then the conditions of any two designs to be comparable were set up; by use of moment matrices. The construction of weighted centroid designs becomes visible. The weights were obtained from an original design, which are used in the construction of the weighted centroid designs. It is shown that the set of the weighted centroid designs constitutes a minimal complete class designs for the Kiefer design ordering. It is also shown that any design that is not weighted centroid design can be improved upon by convex combination of an appropriate elementary design. This study agrees with other studies done earlier for the second -degree Kronecker mixture models by Pukelsheim (1998, 1999). The results obtained were used to get the information matrices and therefore Kiefer optimal designs, hence Kiefer optimality (Gregory. K,2012).

Acknowledgement
I am very grateful for the funding availed to me through the Deutscher Akademischer Austausch Dienst (DAAD). This funding facilitated my undertaking of graduate program and this work could not have been completed without such a generous offer. My sincere gratitude goes to the members of the department for the moral support they accorded me.