An Approach to Uncertainty Identification in the Valuation of Transactive Energy Systems

Transactive energy (TE) systems enable active market participation of demand-side generation assets and distributed loads. Each energy-related transaction involves value exchange between the parties [1]. Within this context, transactive systems are a means for revealing and coordinating the flexibility of distributed energy resources, including responsive building loads, to provide valuable energy services to the grid. Such a mechanism enables the provision of other valuable, nongrid energy services to and from buildings, their owners, and occupants.

Cost-benefit analyses and integrated resource planning (IRP) are common types of valuation methodologies existing in the energy domain. While cost-benefit analyses deals with comparative analysis yielding a go/no-go outcome, an IRP guides the process of optimal portfolio selection to meet future needs. An IRP models growth and dispatch based on load and price projections. For example, leading utility companies use IRP process by defining a core- and sensitivity-case. Then, price forecasts are developed for the planning period. Monte Carlo simulations are performed for a long planning horizon (e.g., 20 years) to develop resource portfolios based on the cases. The process ensures recommended portfolios meet renewable requirements. The proposed valuation methodology supports borrowing results from IRPs. IRPs make infrastructure level analysis, while intricate details at building energy level are managed using co-simulation platforms (e.g., combination of power flow analysis, building models, and markets). The proposed methodology acts as a postanalysis method to synthesize information from all sources of analysis and present them at a various levels of detail sufficient enough to inform analysts as well as managers. The methodology also supports quantification of critical valuation metrics (e.g., energy unserved) as part of risk-informing a decision maker.

Valuation in the context of transactive systems refers to the estimation of accumulated business value to each stakeholder during the process of executing transactions. Sophisticated devices and strategies such as distributed energy resources (DER or DR) and building energy demand management are key enablers of transactive energy systems [2]. However, the mere availability of these mechanisms does not warrant widespread adoption by a savvy community. The stakeholders are interested in tangible values the technology advancements bring to their business, especially when their adoption requires investment decision making.

If conventional methods are considered as a base case and transactive techniques as a test case, then a valuation exercise enables the estimation of net benefits and impacts of choosing one over the other to satisfy a business objective. An important aspect of a valuation exercise is the choice of appropriate operational metrics and associated activities (models) that yield these metrics. Hammerstrom et al. studied the relationship between these principles during the design of a basic valuation process as shown in Fig. 1 [3]. The choice of operational metrics is guided by and tied to the operational business objective. Apart from the operational metrics for decision making, the valuation model itself may be composed of several support metrics.

Like all models, valuation models require inputs, perform certain activities and produce an output, which in this case happens to be a metric for system impact measurement. Inputs dealing with historical data and forecasting are associated with uncertainties [4]. The former leads to variabilities while the latter, in addition, leads to epistemic uncertainties. There is also uncertainty associated with operational reliability of field equipment given their aging characteristics and influence of extreme events. These uncertainties then propagate through valuation activities and manifest themselves in the outcome metric designated for decision making. The accuracy of either type of uncertainties can be improved, but neither of them can be eliminated, unless some simplifying assumptions are made, more data is collected or better knowledge is acquired. These uncertainties present three primary challenges: namely, how are: (a) uncertainties identified, (b) uncertainties treated, and (c) decisions made in light of such uncertainties. This paper provides guidance to a valuation analyst regarding uncertainty identification and its treatment in support of decision making.

The rest of the paper is organized as follows: the Uncertainty Identification section discusses need for identifying and characterizing uncertainties in a valuation process. The peak heat day scenario is introduced as a case study and then evaluation and quantification mechanism for the scenario is described. A discussion follows before concluding the paper.

A valuation exercise should systematically and comprehensively analyze base and test case scenarios to identify key uncertainties and their sources. This helps in understanding the implications of these uncertainties and in formulating solutions to address them. Decision trees and simplified bowtie diagrams have been advocated by some authors for uncertainty identification [5,6]. These have not been found very compatible for a valuation exercise. Bowtie diagrams use a combination of fault tree and event tree diagrams. However, valuation does not necessarily always describe failures. Decision tree diagrams assist in uncertainty classification rather than identification. The authors in this paper describe a deductive reasoning approach with modifications to Bowtie diagrams as an alternative strategy for uncertainty identification in the valuation of transactive energy systems.

Fault trees are commonly used in engineering failure analysis to deductively analyze causes and event sequences leading to a system failure event (called a top event) [7]. The potential root causes are captured as the leaf nodes and contributing causes are identified through intermediate nodes of the tree. The top event can be quantified by assigning frequencies and probabilities to leaf nodes and evaluating the overall system using Boolean logic [8]. An attractive feature of these trees is they can be zoomed-in and zoomed-out depending on the given scenario's desired granularity. This nicety allows for expanding the entire tree up to the level of minute details or collapsing them to a level sufficient enough to get an overview of the problem [9].

Fault trees, with an appropriate name, can also be utilized for general purpose problem solving [10]. In the context of valuation, system objectives need to be fulfilled and the methodology should allow for estimation of an associated impact measure (top node). The tree should allow for enumeration of alternative solutions such that the leaf nodes (basic nodes) can be interrogated for presence of uncertainties. Uncertainties arise in the basic nodes, propagate up the tree along the intermediate nodes, and get reflected in the impact measure. With an appropriate input quantification and evaluation scheme, the “solutions” tree shall enable an analyst to estimate the impact measure for base and test cases so that net benefits and impacts to each stakeholder can be estimated. A quantification scheme will further allow for estimating aggregated uncertainties at different levels of the tree—basic, intermediate, and top nodes. Fault trees typically support many types of Boolean gates (AND, OR…) with unique symbolic representations for each gate. In this paper, no specific relevance is given to Boolean logic or respective symbols. The nodes are annotated with simple algebraic expressions toward quantification of the impact measure.

Consider the “solutions” tree in Fig. 2 where G001 represents the impact measure associated with a system objective. The objective is achieved through three alternative solutions. Solutions A and B are assumed to be part of the base case and Solution C is the test case to be evaluated. The test case is denoted by the suffixed asterisk (*). The reason both base and test case scenario solutions are maintained in the same tree is base and test cases can act as complements rather than substitutes for each other [11].

The solutions tree approach is itself subject to completeness uncertainty [12]. There can be a difference in comprehensiveness between a tree built by a single analyst and a team of analysts collaborating. There is also a modeling uncertainty associated with the solutions tree approach. The source lies in the fact that no two analysts are likely to model a use case in the same way and think about the problem alike. This uncertainty can be addressed to an extent by developing guidelines and rules around its construction. Collective brainstorming and peer review can address this uncertainty.

Not all leaf nodes necessarily represent uncertainties. Some of them are likely to be constants. Yet, others are likely to represent stochastic variability captured from historic data amenable to data fitting. Time series data is also a likely candidate for the leaf nodes. Some of them reflect upon time varying growth such as gradual increase in installed-base over different planning horizons. Finally, certain variables are subject to epistemic uncertainty that arises due to either lack of knowledge or experience. These should be characterized using expert elicitation techniques combined with quantification schemes that support both subjective and evidence-based assessment.

The tree should be constructed such that the solution set is collectively exhaustive including base and test case scenarios. They need not necessarily be mutually exclusive as the case scenarios are in some sense likely to be correlated. However, within an individual case, it is recommended to have independent solutions. In case any two solutions within a particular case are correlated, the nature of these dependencies should be documented and where possible, reflected in the evaluation of the impact measure. It is convenient to group similar solutions together as a subtree to support logical structuring of the tree.

The top node of the tree represents the impact measure and is in itself a valuation metric. As the solutions tree is built and leaf nodes are identified, these are themselves likely to be fundamental metrics or inputs to the valuation model that support the evaluation of the impact measure. These could be constants, aggregated quantities or even results of a simulation. Thus, the methodology supports the comprehensive tabulation of all metrics that contribute to the impact measure. Note that there could be more than one impact measure to be optimized for a scenario. In that case, multiple solution trees would be needed. The decision maker will have to rely on a combined set of impact measures. In some instances, weighted functions serve to bind multiple impact measures into a single decision-metric [13].

The solutions tree can be constructed for qualitatively brainstorming available solutions and to identify uncertainties associated with those solutions. The added benefit is the ability to quantify the tree for valuation purposes. Each solution can be designated with a variable name so that an uncertainty distribution can be assigned to them. Functions representing the impact measure as a function of the identified variables should be formulated such that the measure is distinguishable across base and test cases. The tree should continue to grow breadth and depth-wise until uncertainties in the leaf nodes can be clearly delineated and characterized. Stakeholders are identified and listed beside the impact measure. This is also a placeholder to annotate net benefits and impacts to each stakeholder as a function of the operational impact measure. However, for valuation purposes, the tree can be solved or quantified twice—once for the base case and again for the test case such that the test case variables are set to zero while solving the base case. Likewise, one or more or all of the base case related variables may be set to zero while evaluating the test case. The test case could be all inclusive of the base case or a disruptive solution on its own.

The solutions tree can be readily converted into an influence diagram for mathematical expression evaluation and quantification through Monte Carlo simulation based on efficient sampling techniques. The simulation is algebra of uncertain variables laid out in the form of an analytical expression. Some of the variables may themselves be populated resulting from a domain simulation (e.g., large commercial building energy consumption simulation using EnergyPlus).

Real-word test case deployment requires feasibility studies and thorough validation through valuation, simulation and field demonstration exercises. Such studies can prove time and resource intensive requiring multiplatform integration. The advantage of the solutions tree approach is in performing a first cut feasibility analysis of the test case before delving into detailed design and implementation. This is typical of engineering failure analysis conducted in the nuclear, aerospace, and chemical industries. Fault and event trees are constructed and maintained throughout the design, operations, maintenance, testing and decommissioning life cycle stages of a device or facility.

The solutions tree approach serves well as an integration platform. The leaf nodes can be assigned inputs resulting from a simulation study. These studies could have their roots in domains as varied as the power grid, DERs, markets, and building asset management. If simulations lead to time varying results, the impact measure is then a function of time and the valuation exercise reflects time value of base and test scenarios. Uncertainties in simulation results propagate up to the impact measure. The time evolving trajectory of the impact measure for base and test scenarios incorporating these uncertainties assists in decision making.

The system impact measure and stakeholder specific inputs together can inform net benefits/impacts to the stakeholder. Unserved energy translates to lost revenue to a utility at the rate of the marginal electricity price. The same metric equates to lost production revenue for a commercial vendor or profits otherwise earned by a manufacturer [14]. The loss to a residential customer is not as straightforward requiring smarter methods to effectively price occupant discomfort.

e3 value diagramming is instrumental in capturing the actors, activities and value exchange objects concerned with a valuation exercise [3]. It gives a holistic qualitative picture of the value flow throughout the system. The solutions tree approach is a preliminary analysis to identify high level activities and valuation metrics. The outcomes can then be leveraged to build linkages and complete an e3 value diagram. Both approaches are complementary and support the overall objective of energy system valuation.

The smart grid interoperability panel (SGIP), a nonprofit consortium of representative stakeholders, has identified six landscape scenarios where transactive energy can be potentially applied to mitigate present day energy management challenges [15]. The first of these scenarios is called peak heat day and energy supply use case (SGIP-1). The system objective is to manage the demand either through reduction or a commensurate increase in generating capacity. A critical system impact measure is the ability to minimize unserved energy. The operational means to achieve this objective is to adopt either traditional management techniques like curtailment or a sophisticated mechanism like transactive interaction. In either case, first-tier DER is assumed to be available to increase generation capacity. First-tier DER technologies are generally commercially deployable without transactive mechanisms in place (e.g., rooftop solar panels). Second-tier or back-up DER generally require greater capital investment and may not be readily commercially available in most markets (e.g., commercial scale battery storage capability). The assumption is that this increased capacity does not yet cater the peak demand. In this paper, the traditional solutions are referred to as the base case and transactive solutions as the test case.

Base case involves the application of curtailment methods to reduce demand and relieve the grid of excessive burden. A percentage of this curtailment comes from invoking contracts signed between a utility and its customers. A typical contract states that an advance notice will be provided to the customer concerning the timing and length of curtailment in case of a contingency. Curtailment as part of this contract is at the discretion of the customer, i.e., the customer receives an incentive upon volunteering to have their energy curtailed. Other extreme measures when this step is inadequate include forced curtailment and expensive imports from neighboring jurisdictions.

The test case concerns invocation of transactive operations on the market to incentivize: (a) increased DER supply and simultaneously and (b) lowered demand. Note that this mention of DER refers to resources beyond the first-tier, potentially the ones at marginal pricing within the DER structure. An attempt at reducing the demand can be made by encouraging either: (a) reduced usage or (b) a load shift to off-peak time. The final resort would then be to default to the base case. This situational contingency could happen due to: (a) nonstrategic incentivization of the transactive scheme, (b) misapplication of the transactive operation, (c) operational failure of the transactive system, or (d) customer indifference to the incentivization, or a combination thereof.

The scenario of interest is the proper management of a peak heat day when supply falls short of demand. The system objective is to minimize this shortage. The authors consider the use of unserved energy as a valuation metric to gauge the system impact following the application of base and test cases. The metric facilitates the translation of system operational impact measures into lost revenue as it pertains to each of the stakeholders. At a minimum, these stakeholders are utilities and commercial owners.

The valuation exercise gathers key inputs from base case and test case scenarios, and executes relevant models with these inputs to quantify the identified valuation metric(s) for net benefit/impact estimation. When a solutions tree is exhaustively constructed for this scenario, the leaf nodes represent required inputs for the respective valuation models. These inputs can then be inspected for uncertainties.

The solutions tree approach described earlier will be constructed in this section for the peak heat day scenario. The scenario analysis will serve as example guidance for uncertainty identification. At a high level as seen in Fig. 3, the possible solutions to peak heat day management are either doing nothing, increasing the capacity or decreasing the demand. The last two solutions can be complementary and be undertaken simultaneously while the first option basically provides an estimate of the deficit in the available capacity. The net deficit following capacity additions and demand reduction is the residual unserved energy, depicted at the top of the tree as an impact measure. Implications to each stakeholder as function of the marginal electricity price are shown as impacts. The solutions at the top level are rolled up with high level solutions. There is scope for further scrutiny and tree development.

The unserved energy in the event of not taking any action is the net difference between actual demand (E) and forecasted demand (A). Forecasting requires further inputs and detailed modeling; however, the node is left as an undeveloped leaf node. This level of detail is assumed to be sufficient to capture uncertainties at a broad level. The main uncertainties stem from forecasting errors and epistemic sources (e.g., weather variability and incomplete observations). On the contrary actual demand is a constant. Hence (E − A) is an uncertain quantity requiring interventions to stabilize the power grid and provide reliable services to customers. This subtree is illustrated in Fig. 4. Note that estimating the residual unserved energy is essential in both base and test cases.

In the event of unavailable capacity to meet demand, avenues to reduce the demand must be explored as illustrated in Fig. 5. The base case techniques are to either curtail the demand or manage the loads without active market participation from the customer's end. The test case is expected to achieve the same through increased customer involvement in deregulated markets using DERs and the transactive suite of techniques.

Load management in the base case is achieved through demand response management programs (M). These include but are not limited to dynamic pricing and direct load control. For example, time of use pricing with advanced metering infrastructure encourages lowering demand during peak periods. Some of these solutions involve uncertainty in customer enrollment, pricing structure, and advanced metering infrastructure reliability.

The use of the TE mechanism is summarized in the SGIP-1 narrative as—“[to] build upon DR by using TE's ability to support dynamic price structures that respond to current or anticipated grid conditions. This approach incents loads and DER to participate in overall lowering demand on the grid. Additional capacity is incentivized through compensation that appropriately values the reliability provided by DR participation,” [15].

Load shift (H) and load reduction (K) solutions are designated as transactive techniques in the solutions tree to reduce demand. DERs enable market participation through construction of supply bidding curves for the energy generated. Devices enable the same through a compilation of demand bidding curves. There is element of aggregation done on bids received across a group, such as feeders. Clearing price is formulated at the intersection of these bids [16].

Transaction-based building energy systems can facilitate demand reductions that extend well beyond grid operations by enabling the exchange of information between buildings, third-party service providers, and other entities. These exchanges are based on electronic and primarily automated transactions, delivery of services, and settlement for services rendered. More efficient and effective provisions of building services result in better control, smarter buildings, reduced energy consumption and costs, co-investment opportunities, enhanced monitoring and verification, and enabling new products and services that save energy and increase economic productivity. On a more granular level, within the building environment the use of automated smart devices and autonomous decision making helps reduce demand based on end-user preferences. If appliances are responsive to incentives, usage can be either preponed or delayed thus shifting the load. If thermostats actively participate in the market, lowering the temperature setpoint helps in load reduction. The key to success of these schemes is in the attractive and accurate exchange of value objects such as pricing structure and quantity of the underlying commodity required.

Though automated, the transactive mechanism is largely voluntary because of active participation from the demand-side. In this sense, transactive management is a generalization of demand response management and involves a broader set of uncertainties i.e., demand-side uncertainties are compounded with supply-side uncertainties. The exact transactive mechanism may vary from one scheme to another (e.g., transactive energy market information exchange, pacific northwest (PNW) smart grid demonstration project, PowerMatcher) [17]. Hence sources of uncertainties should be analyzed according to the chosen transactive market mechanism.

Curtailment is a solution to eliminate increased demand. It can be either voluntary (G) or forced (N). In the context of the subtree shown in Fig. 6, curtailment is a base case solution being practiced by most jurisdictions today. An advance contract is signed between utilities and targeted customers willing to participate in voluntary curtailment.

In the event of contingency, the contract is invoked and cooperating customers are given a credit for being curtailed. Forced curtailment is a last resort to ensure grid stability. Excess demand is curtailed, but it can bring about heavy monetary losses to the utility and is not generally considered a solution for reducing the unserved energy. The forced curtailment solution is included here for completeness and not for the purposes of impact measure evaluation. This scheme is to ensure impact to the utility is properly accounted for at the top of the tree. There is uncertainty in voluntary curtailment owing to the dependence of the utility on the customer at the time of contingency despite a contract being signed. As the contingency itself is a rare event, it would be a challenge to probabilistically characterize this uncertainty. A survey or polling of the customers for their willingness to voluntary curtailment given the time, duration and magnitude of the given event can help address the characterization process.

The available options to increase capacity are captured in a subtree shown in Fig. 7. Tapping first-tier DER (F) is assumed to be common to both base and test case scenarios beyond which different management mechanisms are, respectively, applied to mitigate the peak heat day scenario. The other option to increase capacity in the base case is to import energy from another jurisdiction (L). This case is included here for completeness but not for evaluation purposes. The reason for this exclusion is zero residual unserved energy that would result reflecting zero lost revenue to the utility. However, the utility does lose some revenue due to expensive import prices. The transactive mechanism can contribute to increased capacity through tapping backup DERs. Weather forecasts, market pricing, DER owner involvement and end-user participation and preferences all contribute to the sources of uncertainties.

This section applies the solutions tree evaluation approach to the SGIP-1 use case. All the identified solution variables are measured in the units of MWh. The amount of additional capacity tapped or demand reduced and the duration for which the activity is undertaken together determines the total energy for that particular leaf node. The nodes can be annotated with the assigned numbers or distributions along with the duration.

The unserved energy in the absence of any mitigation depends on the following variables:

• forecasted energy demand (A)

• actual energy demand (E)

and is quantified in both base and test cases as the difference between the two i.e., (E − A) assuming E > A. For the reduced demand subtree, the following are the relevant variables:

• energy demand reduced via time-varying rates under a demand response program (M)

• energy demand reduced via voluntary curtailment (G)

• Load shifted through transactive mechanism (H)

• Load reduced through transactive mechanism (K)

The unserved energy can be offset in the base case by (M + G) or in the transactive case by (H + K).

Capacity addition uses the following variables:

• First-tier DER (F)

• Backup DER (J)

The next steps in the process include data collection and characterizing the above random variables with appropriate distribution fits to the data. The impact metrics can be conveniently converted to loss estimation in terms of dollars per MWh relevant to each customer. A risk-informed approach to decision making then involves comparison of loss distributions taking into consideration results of sensitivity study, individual node importance, and stakeholder specific concerns. The entire tree is presented in Fig. 8 for an overall flow of readability.

Often models are overly simplified to avoid identifying and addressing uncertainties. It is important to understand that the evaluated metric in the form of a single number, so called point estimate, is not adequate to support decision-making. An example of a point estimate is the median. Historical variation and forecast errors contribute to point estimate insensitivity. However, the uncertainty distribution around a point estimate captures information relevant for business decisions. The width of the distribution informs variability in the impact measure and hence reflects the inherent risks to the business objectives. For example, base and test case scenarios could have the same representative point estimate but the test case likely has a wider uncertainty than the base scenario. A decision-maker tends to choose the alternative that has relatively lesser uncertainty. But, width alone is not the deciding factor. Skewness or the asymmetrical nature of a distribution also plays an important role in decision-making. Left tailed (negatively skewed) distributions carry higher probability of occurrence toward the right. For example, consider the uncertainty in unserved energy. If a mitigation mechanism allows for a right tailed distribution, it would be preferred to a competing negatively (left) skewed distribution because larger values of unserved energy carry smaller probability.

In summary, once a solutions tree is exhaustively constructed for a postulated scenario, a valuation analyst can assign variables to each leaf node solution. The analyst can work their way up the tree to formulate two mathematical expressions: one for the base case and the other for the test case in support of valuation. The uncertainties associated with the leaf nodes should be tabulated along with the implications, sources and assumptions. Appropriate statistical distributions and constants should be assigned to each of the variables. Risk analysis software can be used to simulate random variables with the objective of quantifying the value of base and test case scenarios. Where applicable, variables can be selectively chosen for sensitivity studies. An example would be a conditional growth of x% in the installation of DER in the next 3 years.

Valuation supports comparison of base and test case scenarios. Transactive energy systems bind together bulk supply, DER supply, demand (e.g., building energy system loads), and markets for effective energy management. Valuation of transactive energy systems assists decision-makers in making a business case for deferment of costly investments, engagement of more DERs and increased customer participation. This paper discussed an approach to systematically analyzing complex scenarios for possible solutions and to facilitating uncertainty identification. The quantification scheme for the described approach lends itself readily for valuation. This method is complementary to simulation and field demonstration studies. The smart grid interoperability panel peak heat day scenario was considered as an example to illustrate the proposed approach.

The work was performed at Pacific Northwest National Laboratory, operated by Battelle Memorial Institute for the U.S. Department of Energy.

• U.S. Department of Energy Office of Electricity Delivery and Energy Reliability and Energy Efficiency and Renewable Energy Building Technologies Office (DE-AC05-76RL01830).