Markovian Decision Modeling in Dam Projects-Niger Delta River Basin

This paper studied simulation modeling in Markovian Decision theory and its application in decision making as well as planning in water resources and environmental engineering. The research objectives deals with the multiobjective values of a River basin for its wide range of purposes such as Economic Efficiency, Regional Economic distribution, State Economic distribution, Social Well-being, and Environmental Quality control. In line with foregoing objectives, the researchers aim at achieving the following: (i) Measures the magnitude of the difference between alternative actions (ii) to present a framework for considering decision making under uncertainty. (iii) to evaluate the optimal policy or strategy or action that maximizes the expected benefit in the River Basin within the available limited resources and funds over the planning period of a course of action or alternatives. The Methodology applied involved Markovian decision model method for River basin. Data collection was based on technical literatures from books, journals, and news papers, River Basin Engineering Development, Parastatals. The analysis and presentation of results were based on simulation of Markovian Models. Furthermore, Contingency association, Chi-square, Pearson Product Moment Correlation were carried out as interaction, reliability and validity tests. However, simulating the river basin variables using Markov chain Homogeneous analysis and policy iterations resulted to a decision policy of allocating resources to the river basin objectives based on a federal government budgetary appropriation of 100 billion Naira. In conclusion the model had policy decision made as follows: Economic Efficiency [64%], Regional Economic Distribution [9%], State Economic Distribution [19%], Social Well-Being [5%] and Environmental Control [3%] [see Figure 1 and 2]. The results indicate that Markov Chain can be successfully applied in optimum policy investment decision making in multi-objective water resources management.



To evaluate the optimal policy or strategy or action that maximizes the expected yield of the River Basin purposes Area of Study:The study area is the Niger delta river basin that lies between 6.83N and 6.75E; 5.38S and 5. 37W. Niger delta basin development authority is a service -oriented organization that is positioned to meet the water requirements of stakeholders in the most satisfactory and cost effective manner, while ensuring good quality and sanitation and paying adequate attention to preservation of the ecosystem, using proven technology and a well-motivated force[NDBDA MISSION]. In terms of geographical coverage it serves Rivers state, Bayelsa and Delta states. The three states have an estimated population of 10.7 Billion people.

Methodology
Under this section, the researcher identified estimation methods of the two major parameters of river basin indicators and the Markova method of application as follows: Estimating Multipurpose Benefits There are six data categories that structure the multipurpose benefits framework. These categories are referred to herein as "uses", and they represent a culmination of operations and services made possible due to existence of a reservoir. These uses are broadly classified to identify categories associated with a reservoir project, and serve as a foundation for assessing collective and inter-dependent relationships (Marisol Bonnet et al, 2015): i. Hydropower: Operation and use of generating facilities and/or equipment for producing power by the sole source of water. ii. Flood Control: Dams that facilitate the prevention and/or lessen the severity of flood damage to valuable resources within a flood basin. iii. Water Transport&Navigation: The operation and control of locks to facilitate the transportation of goods via inland waterways. iv. Recreation& Tourism: The use of water bodies (reservoirs or rivers) for physical and recreational activities (boating, fishing, swimming, etc.). v. Water Supply: Public and private withdrawals of water used for consumption, municipal, and industrial needs. vi. Irrigation: The withdrawal and use of water from reservoirs to meet the needs and requirement for crop and plant irrigation to sustain growth and production. Based on the availability of both public and proprietary data, the following represent the methodologies used to compute the economic benefit of each multipurpose use.

Markovian Simulation Method
The method of Markov chain applied in this research work is homogeneous Markov chain one that does not evolve in time; that is, its transition probabilities are independent of the time step n. Then we have the "nstep" transition probabilities as stated below: and we have Now we can define a theorem. Theorem.Chapman-Kolmogorov equation.

Proof.
We can write this as a matrix for convenience:
Proof. Chapman-Kolmogorov in matrix form gives us Several definitions A Markov Chain iscompletely determined by its transition probabilities and its initial distributionn. An initial distribution is a probability distribution such that A distribution is stationary if it satisfies π = πP. The period ofstate i is defined as that is, the gcd of the numbers of steps that it can take to return to the state. If di = 1, the state is aperiodic-it can occur at non-regular intervals. A state j is accessible from a state i if the system, when started in i, has a nonzero probability ofeventually transitioning to j, or more formally if there exists some n ≥ 0 such that We write this as (i → j).We define the first-passage time (or "hitting time") probabilities as that is, the time step at which we first reach state j.We denote theexpected "return time" as A state isrecurrent if (and transient if the sum is greater than 1). It is positive-recurrent if µii < ∞. That is, we expect to return to the state in a finite number of time steps. Fundamental Theorem of Markov Chains Theorem. For any irreducible, aperiodic, positive-recurrent Markov chain P there exists a unique stationary distribution {πj, j ∈ Z}. Proof. We know that for any m, If we take the limit as m → ∞: This implies that for any M,

Data Estimation, Analysis and Optimization
Determination of benefits to purposes under various objectives in a multi-purpose/multi-objective Water Resources Project Planning: At the onset of planning of multipurpose water resources project, it is necessary to declare the objectives against which efforts is being geared for their achievement, this serve as a criterion for measuring the projected end product of the planning process. The main objective that can come into play in a multi-objective water resources development are (1) economic efficiency (economic optimization), (2) Regional economic redistribution, and (3) Social well-being. Any other objective can be incidental on the above three.

Application of Markov Theory in Multi-Purpose Multi-Objective Projects Optimization
Let's consider Federal Government Allocation to Niger Delta River Basin whereN100 billion is to be spent on a multi-purpose/multi-objective water resources development project. The purposes of interest are Navigation, Tourism, Flooding, Hydro-electric power generation and water supply. The objectives to be simultaneously achieved at optimum level are economic efficiency, regional economic redistribution, State Economic distribution, social well-being and Environmental quality. The problem then becomes how to apportion the N100 billion developmentfund among the various purposes so as to optimize the objective even under the worst situation of conflict. A benefit study of the five purposes under each of the five objectives was carried out. The results being the figures as shown in table 5.1. What we have by the table is basically a Matrix situation that satisfies the homogeneous Markov chain.

Table 1 Benefit to N100 Billion under various objectives [N X 10 9 ]
Table 5.1 above is in matrix form and is converted into homogeneous transition or stochastic matrix to satisfy Markov Chain process where the probability Pij must satisfy the conditions: ∑ Pij = 1, for all I; Pij ≥ 0 for all I and j Converting Table 5.2 to a linear equation as following: The above Matrix problem can be solved from the maximize point of view with the understanding that all purposes should be undertaken at positive level even under the worst circumstances or condition. Let probability π1 represent Navigation Let probability π2 represents Tourism Let probability π3 represents flooding Let probability π4 represents Hydropower And Let probability π5 represents Water supply P = These Probabilities in the matrix were calculated by the formula: Where Nijis the number of observed transitions from state i to j.

Markov Chain Analysis
The equations having satisfied Markova homogeneous chain are analyzed by Markov steady state. There two methods for solving the infinite-stage problem. The first method calls for evaluating all possible stationary polices of the decision problem. This is equivalent to an exhaustive enumeration process and can be used only if the number of stationary policies is reasonably small.The second method, called policy iteration, is generally more effective because it determines the optimum policy iteratively.Conversely, the second method was adopted for this research work,     Table 4&5, it appears to be the same i.e. the iteration has reached a steady state and can no longer change; this can also be called optimum solution or values.  should be not less than five observations in any one cell. (f) Not more than 20% of the expected frequency should be less than 5. The X 2 can be used to treat data which are classified into nominal, non-ordered categories; it can also be employed with numerical data. The researcher may wish, however to analyze such data with more powerful parametric test. But for nominal data, few alternatives to X 2 analysis exist. The basic computation equation for X 2 is given below: It should be noted that whenever X 2 is calculated from (1 by 2) or ( 2 by 2 ) cell tables( instances in which the degree of freedom is one ) an adjustment known as Yates correction for continuity must be employed. To use this correction a value of 0.5 is subtracted from the absolute value (irrespective of algebraic sign) of the numerator contribution of each cell.

Contingency Coefficient, C is given by
Where C = Contingency Coefficient X² = Chi-square N = Grand total of subjects or cases 5

Correlation of Attributes
The degree to which one of the attributes depend upon is associated with or related to the other attribute is referred to as correlation of attributes. In the k x k Contingency the correlation of attributes, r is given as : For a 2 X 2 table the correlation attribute is called tetra choric.

Contingency and Reliability Test
Contingency and reliability in this paper is another alternative method of testing null hypothesis, the paper assesses the relationship and test the null hypothesis on: "There is a relationship between the Watershed Purposes and Objectives"

Equation 19
Equation 20 Equation 21 Civil and Environmental Research www.iiste.org ISSN 2224-5790 (Paper) ISSN 2225-0514 (Online) DOI: 10.7176/CER Vol.11, No.2, 2019 98 Step I: Calculation of the expected contingency table using the formula: Where I = is the i th and J = is the j th column Below: StepII: Computation of Chi-square using the formula:

Presentation of Results.
The Contingency of the raw data is = 0.63. The correlation of attributes of the raw data = 0.3. The X 2 value 23.63436176is interpreted from the X 2 table of probability values at 0.10 level of significance. The degree of freedom necessary to intercept X 2 values are always determined from the frequency table by the number of rows minus one times the number of columns minus one (r-1)(c-1) i.e. (5-1)(5-1) = 16 -Since the obtained X 2 value of 23.63436176is less than the critical value of 32.000, therefore the Alternate Hypotheses is accepted. i.e.: X 2 (23.63436176) ˂ X 2 0.10 (32.000). Therefore the Alternate Hypothesis'saccepted, a clear indication that there is a relationship between the watershed purposes and the Objectives/Benefits. -Therefore there is relationship between the state of the system (Dam Purposes) and the Dam Objectives.
-The Chi Square was not based on a fictitious data, in the case of Markov Decision Modeling in Niger Delta River Basin.

6.1analysis of Variance[Anovar]
The Pearson Product Moment Correlation Coefficient often referred to as the Pearson R tests, is a statistical formula that measures the strength between variables and relationships. To determine how strong the relationship is between two variables, you need to find the coefficient value, which can range between -1.00 and 1.00. The computations are done as shown in Table 11using equation 23 and results displayed graphically in Figure 3 The analysis of variance in this reseach work can be done using the following methods:  Table 11 ] The initial benefits Iteration and 50 th Iteration benefits values were correlated using Pearson moment correlation coefficient formula and r was determined as 0.9851 in table 11 and the graph represented in figure 3.

Model Validation
The initial iteration and 50 th iteration were plotted, as in column 2and 3of table 11 for the validation of the model.Therefore R = 0.985

Conclusion and Recommendation
Based on the findings and conclusions reached on the study the following recommendations are made: Niger delta has more water available; therefore it is recommended that Hydropower in this region should be considered and encouraged because of it immediate and long term benefits when compared to gas powered electric plants. Also clean environment should be embraced for a healthy land, water and air; and in turn increase the level of tourism as well as reduces flooding caused by environmental abuse.

Contribution to Knowledge
The study can provide an organized baseline for future work, mainly in obtaining superior estimates for institutional water use and planning by the aid of Markovian decision theory. However, the findings of the study can be vital input into the demand management process for long term sustainable water supply within Niger Delta River Basin and beyond.