A Policy and Evaluation Framework for Sustainable Transitions - An Energy Policy Approach

Joana Sousa ^*

1. Lusófona University, Faculty of Natural Sciences, Engineering and Technology, Lisboa, Portugal

* Correspondence: Joana Sousa

Academic Editor: Paweł Ziemba

Special Issue: Effective Leadership and Governance in Sustainability Transitions

Received: May 15, 2023 | Accepted: January 08, 2024 | Published: January 15, 2024

Adv Environ Eng Res 2024, Volume 5, Issue 1, doi:10.21926/aeer.2401003

Recommended citation: Sousa J. A Policy and Evaluation Framework for Sustainable Transitions - An Energy Policy Approach. Adv Environ Eng Res 2024; 5(1): 003; doi:10.21926/aeer.2401003.

© 2024 by the authors. This is an open access article distributed under the conditions of the Creative Commons by Attribution License, which permits unrestricted use, distribution, and reproduction in any medium or format, provided the original work is correctly cited.

1. Introduction

Presently, human activities correspond to an open cycle that starts from an initial environmental balance and ends with a condition of environmental imbalance [1]. Despite the scientific evidence highlighting the drastic need for global emissions reductions, anthropogenic emissions are still increasing [2,3]. According to the IPCC report on greenhouse gases (GHG), increasing rates have been caused by human activities over the past six decades, namely: energy use, land-use and land-use change, and patterns of consumption and production [3]. Climate destabilization may be considered a public welfare problem on a global scale that if not properly addressed may become an economic, social and humanitarian crisis [4,5]. Technological innovation is considered a key driving force for sustainable development in transformation processes. However, the COVID-19 pandemic technology push has not been accompanied by social well-being but by a pushback on the implementation of the Sustainable Development Goals (SDGs) [6,7,8].

An increasing number of authors consider societal and environmental goals as legitimate drivers for innovation policy [9,10,11,12,13,14,15,16,17]. According to Borrás and Schwaag Serger (2022), the emerging policy rationale of “transformative research and innovation policy” suggests that research and innovation (R&I) should address grand challenges and foster the transformation of complex socio-technical systems inducing different patterns of production and consumption in society [18,19,20]. Current lifestyle relies on a continuous demand for energy causing this sector to be a major responsible for the CO₂ emissions. For this reason, socio-technical systems transformation cannot be disconnected from energy policies which target directly and indirectly energy systems.

In opposite to the past focused on optimizing the innovation “ecosystem”, in the third generation of innovation policy a discussion on directionality has arisen [9,16]. The last innovation policy frame is not just about optimizing the innovation system towards economic competitiveness. Innovations should be efficient within the strategic direction as guiding processes of transformative change towards societal needs [16,21]. The innovation policy third generation has a wider perspective than previous generations and focuses on innovation processes to solve societal systems problems [9]. According to Parks (2022), directionality requires demand-side actors’ participation in innovation processes. A key challenge to integrate directionality in the policy requires understanding the actors involved in innovation processes and the actors that determine the change in the direction and how the power relations are changing [15,22].

Accelerating the pace of energy system decarbonization will require an innovation policy that gives a new direction to the rate of innovation, development, and dissemination of desirable energy-related technologies, with radical transformations of business plans, public and private investments, and the reskilling of the workforce [4,23,24]. However, policy goals should also include social participation named as the “quadruple helix” by [22]. This includes broader participation of actors, stimulating demand side and public demand as a driver of transformation, which includes not only citizens, but also civil society organizations, local governments, and infrastructure companies [9,10,11,12,13,25,26,27,28,29]. Energy policies may significantly benefit from the wider perspective of innovation policy third-generation insights, with a broader focus on the re-configuration of existing technologies and actors. To accelerate the pace of decarbonization of energy systems in addition to technological change, social engagement must be considered, and research and innovation should be complemented by governance capacities and market creation and reshaping [9,21]. The need to redraw the present wholesale electricity market design for demand participation in Europe is an example of market reshaping and social engagement [30,31,32].

Energy has been seen by users as a commodity due to its importance to the modern economy, therefore energy-related policies need to assess not only the economic performance but also the environmental dimensions to achieve sustainable development and economic growth [4,30]. Electrification is seen as a pathway to achieve sustainable development, but while some parts of the electric sector remain as “natural monopolies”, other parts accept competition. This might create conflicts of interest and obstacles to energy policy implementation and decarbonization of the sector. The consequences of monopolies and conflicts of interest within the Chinese electricity industry are stated by Wang and Chen (2012), causing private companies to move away from investing and participating in the market resulting in a deadweight loss [33]. Potential conflict of interest among different actors concerning demand response implementation was studied by [34]. Failures are not just resultant of monopolies and conflicts of interest but also due to the non-equilibrium condition between systems’ supply and demand sides. I.e. in some electricity wholesale energy markets end-consumers cannot actively participate (they can only choose the electricity retailer company), by introducing consumer participation the energy prices would be expected to decrease. Demand participation in energy markets may exemplify how transformative change is not purely technological innovation but could also be about the integration of a range of non-technological innovations [35]. Active demand response participation in electricity markets includes both technological innovation developments (smart metering technology, etc.) and non-technological innovations (demand participation). This demonstrates that the content of innovations may go beyond science and technology and should include experience-based learning or institutional change instead of innovation as traditionally studied [25,36].

Transitions are inherently political processes that cannot be achieved through a single policy or experiment but will require the combination of different policy tools that need to be regularly updated and adjusted according to their results and evolution [20,24]. Innovation uncertainty emerges as a major economic and social challenge, that will require trade-offs between directionality, demand articulation, policy coordination, reflexivity and experimentation with new networks between the state, business, civil society, and new supranational structures ensuring global coordination [7,13,15]. The generation of local employment and opportunities in rural communities located in Portuguese economically depressed areas is a valued aspect by local populations and may be used as a trade-off to increase population acceptance [37]. Trade-offs could result in economic benefits increasing local population welfare and avoiding migration to other geographies.

This research conceptualizes a policy framework that establishes the articulation and interaction among different policy levels, identifying possible failures at each level and developing a policy evaluation framework that considers both the demand and supply sides. The innovation of the work focuses on three major points. First, there is a conceptualization of a policy framework considering different levels adopted from Geels’s (2002) framework [38]. Second, failures were classified according to the levels, thus these may hamper the policy implementation. Third, an evaluation framework was developed to consider social reconfiguration and include some transformative aspects, such as reflexivity, and learning capacity to potentially overcome failures. In addition, this research contributes to the following questions:

• How do policies promoting the transformation of systems - such as energy policies - might interact? 
• How could transformation and innovative aspects be integrated into policy evaluation frameworks to overcome policy failure at different levels?

This paper is organized as such: following the Introduction in Section I, Section II describes the methodology defined, namely the analytical framework, the failures towards transformative policy and the evaluation framework for policies. Section III presents the discussion with an energy policy example and Section IV presents conclusions.

2. Methodology

The present research methodology involves three major parts. It starts with the conceptualization of the policy framework considering its multi-level dimension and what refers to each dimension. Then, there is the classification of possible failures according to each level of the policy framework. In the end, there is the development of an evaluation framework that considers social reconfiguration and includes some transformative aspects such as reflexivity and continuous learning to overcome the identified failures.

2.1 Conceptualization of a Policy Framework

Transitions are complex processes due to a large number of interactions and actors. The prior defined framework for industrial policies identifies some pillars and concepts but does not establish policy articulation and interaction processes. To establish a rationale for a policy framework Pisano et al. (2014, June) and Andersson et al., (2021) research set the basis for this policy framework definition. Through the analysis of Pisano et al., (2014, June) research, three levels were considered: the micro level, the intermediate level or the meso level, and the macro level (Figure 1). In addition, Andersson et al., (2021) research was taken into consideration, thus these levels interact with each other dynamically.

Figure 1 Representation of the framework for transformative policy levels.

2.2 Definition of a Transformative Policy Framework Based on the MLP

In Geels’s (2002) socio-technical regime Multi-Level Perspective (MLP) there is a conceptualization of how transitions occur through interactions among three levels, namely with the interactions and interplay at niches, regimes and landscapes [38,39]. Other authors adapted Geels’s (2002) MLP to perform a distinct analysis. Pisano et al., (2014, June) distinguished different levels of interactions, considering the networks, communities and organizations [40]. In addition, Grin (2010) performed three levels of power corresponding to the MLP transition dynamics [41]. In similarity to these studies, the policy framework developed was based on the MLP with three levels: the policy shape at the micro level, the policy context at the meso level and the policy universe at the landscape level (Figure 1). The policy framework cannot be assessed without considering policy directionality. The differences between Geels’s (2002) MLP and this research are summarized in Table 1.

Table 1 Comparison of concept, levels and process with Geels’s (2002) MLP.

The levels reflect different policy scales and their interactions. The macro level or the policy universe reflects the governmental options that shape and define its policies. At the meso level, the policy context reflects the policy intervention area and limits, the intervention area of the policy. The policy shape corresponds to the micro-level and reflects the policy-specific goals. The policy direction corresponds to the trajectory both of policy context and shape, which act within the policy universe.

2.2.1 Policy Universe (The Macro-level)

The policy universe reflects the governmental options. Governments adopt options that shape and define their policies, thereby influencing the transformation of their economies and societies when pursuing societal goals. Several authors recognize this is relevant to problematize the government’s role when considering effective and efficient industrial policy intervention, recognizing that the problem is not whether to intervene, but how to intervene. According to Grillitsch et al., (2019), the debate about government intervention should not be “how much” states intervene in society, but in which ways should the state intervene and what are the implications of these modes of engagement for society. In opposition to the concept of the neoliberal state, a more active and more embeddedness relationship between the state and other socio-economic actors, constructing new relationships between the state, the market, and civil society, and new forms of pro-active and entrepreneurial state action, is defended by [13]. Barker et al., (2022) argue that government action is crucial to accelerate investment in low-carbon or net-zero technologies by supporting private sector investment, direct public investment or regulation and pushing local governments for greener processes through guidance and events [24]. Moreover, government policies may provide room for experimentation in multiple ways, directly in the form of funding and other forms of support for test and demonstration projects. Designing transformative policies will require coherence and governance capacity to dialogue between different levels of governance and sectors to facilitate feedback loops and learning [14,36,42].

2.2.2 Policy Context (The Meso-level)

The policy context reflects the meso level and is shaped by multi-level scale articulation. Policy coherence is recognized by several authors to be an important factor for its integration [5,9,11,12,13,16,27,35,42].

The policy context limits the intervention area of the policy. It interacts with the policy universe and policy shape. It influences and is influenced by the policy’s shape and direction. Variables such as time, market deployment (such as for new technologies), demand-side (users’ preferences), and geographic scale integration, conditionate the context, and consequently the policy shape at the micro level. Time and scale directly affect the deployment of new technological assets. In the past, policy assumed that technology adoption and capital formation were linear innovation processes and investments, depending on the surrounding social, cultural, technical, financial, and institutional environment. However, failures indicated that an enabling environment is critical [4].

2.2.3 Policy Shape (The Micro-level)

The policy shape corresponds to the micro level and reflects the policy-specific goals. For that purpose, policies must consider clear goals set by multiple actors, measurable targets, close monitoring, proper evaluation and well-designed rules considering a new economic arrangement. Better identification of goals and new indicators (metrics) to monitor them are considered by many authors as key to policy success [5,11,17]. The proper choice of targets and tools depends first and foremost on the societal goals to be pursued, which implies discussing and finding a general agreement about the political priorities to be promoted. However, the interests of a broad stakeholder group to agree on a shared vision consensus may not be required through negotiation between interested parties. Goals set with multi-actor may generate greater legitimacy but it may hamper “Blue sky innovation” experimentation and if goals are too narrow, experimentation failures may emerge.

2.2.4 Policy Direction

Policy direction is not considered as a level. However, the policy direction interacts with the context and the policy shape. Moreover, the policy direction acts within the policy universe level.

Directionality is not linear (x,y) but a complex issue. When considering directionality, a vector trajectory with more than two coordinates should be considered (x,y,z) [18]. Directionality is associated with the plurality of the economy’s trajectories, subject to any given set of productive interdependencies and institutional conditions, within the policy universe and context. Additionally, institutional and political lock-in situations may occur if continuous learning and innovation system structure adaptation does not exist.

2.3 Failures Classification Within the Transformative Policy Framework

In the second part, failures stated in innovation and industrial policy studies were collected. The term “failures” was searched in a set of innovation studies, policy studies and sustainability transitions studies. Failures collection was not limited to a specific area to identify the larger number of failures. However, the purpose of this study was not to perform a systematic literature review on failures, but to collect failures and classify them according to each policy framework level.

The third framing of innovation emerged based on the argument that environmental and social goals can be seen as strategic and dynamic drivers. However, this is coupled with the need to overcome transformation failures. According to Chaminade, C., & Lundvall, B. Å. (2019), market failure is pointed to be the main rationale for innovation policy by neoclassical economists, and government-stronger innovation systems are pointed to be the main rationale for system failures [36]. Both neoclassical and system-based policies have been criticized for being reactive. Addressing market or systemic failures, according to some literature will produce incremental changes, thus grand challenges imply radical system transformations that require a more proactive, entrepreneurial and leading role of the state [36]. In Table 2, failures are classified according to the three levels.

Table 2 Failures classification within the policy framework.

According to the defined framework, the policy universe level reflects policy options that influence the transformation of economies and societies when pursuing societal goals. At this level, there is a close link between the policies that are set by the government and society’s problems. The policy universe level is the broadest level, the failures that occur at this stage may hamper policy success and societal transformation. Different authors reported possible failures that may occur at this stage from public or government failures [11,43,47] to policy legacies from earlier policy approaches [26]. The problematization of the government’s role when considering effective and efficient industrial policy intervention, about where and how governments should intervene may fail due to present or past unsuitable governmental choices. These failures may originate other failures, such as a lack of understanding of societal issues that are ill-defined or insufficiently understood [21], political capture [48], excessive techno-centric standardization processes [49] and lack of policy coordination [13,44,50]. At this stage, the correct identification of societal goals is fundamental for avoiding failures. It is necessary to consider that social needs and problems may evolve with time, and thus, it is necessary to consider continuous policy processes for a balance between the outcomes and the societal needs. More than evaluating policy results and outcomes, the new generation of policies should evaluate the policy’s social impacts.

The policy context is at the meso level and reflects the policy intervention area. The policy context is defined within the policy universe and the policy shape. At this level, most failures are associated with different market aspects: market dynamics, innovation system markets and research and development markets [9,11,26,35,36,43,45,49,50,51]. In addition, demand, supply, demand-supply interaction, experimentation and its articulation failures [9,12,13,43,44,45,49], and other failures in the policy context such as lack of scaling up and diffusion of innovative solutions trapped in their original niches and technology bottlenecks are pointed out by [40,49,52]. The time effect caused by waiting games or time lag may also be considered a context failure [52].

Within the policy shape level, strategy, coordination, institutional power relationships and sector integration failures were identified. Other failures include information asymmetry, network failures, expectations between diverse groups of users and suppliers of new technologies, leadership failure, responsibilities articulation failures, replacement/balancing influence from incumbent actors’ failures, and coordination and interaction failures.

Directionality failures, associated with the policy direction, were identified by several authors, and within learning processes. Learning and reflexivity failures as well as experimentation failures including transparency and feedback failures should consider the entire policy framework, thus, communication, learning and reflexivity are relevant aspects for policy accomplishment.

The policy shape and the policy directionality should be permanently articulated, thus the first is associated with policy-specific goals and directionality to the policy trajectory. Policies must consider clear objectives to set directionally. Both directionality failures and micro-level failures such as existing infrastructure, capabilities, institutional weakness, information asymmetry and articulation failures are associated due to a necessity of articulation between these.

2.4 Evaluation Framework for Transformative Policy

This research’s third stage developed an evaluation framework considering the previously defined policy framework and took into consideration the interactions among the different levels. The policy evaluation was conceptualized considering the complexity of policies and was based on traditional policy evaluation, particularly energy policy evaluation. The intended outcome will require the effectiveness and efficiency of policies through new technologies’ development, new actor-network and institutional reconfiguration. The outcomes must go beyond the economic rationale and address the societal outcomes on a continuous adaptation process that should be done through continuing reflection, learning and monitoring.

Static and inflexible policy roles will open the path to more dynamic and flexible roles. Due to path dependency, failures will persist but with reflexivity, the interpretation of the outcomes will differ significantly. According to Janssen et al. (2022): “transition programmes requires evaluators to (…) account for interactions between instruments, engage (and manage conflicts between) a broader set of stakeholders, and achieve coordination between different scientific and technological fields, policy domains, and sectors” [10]. More dynamic and flexible policy roles will require a more complex policy evaluation. Chaminade and Lundvall (2019), state that mid-step evaluation between overall goals, with possible changes in system directionality and particularly in policy instruments for achieving these goals, must be performed.

Within the evaluation framework, the policy universe level concerning governmental options should tackle social concerns and goals. In the policy framework developed, the policy universe and policy context levels are constantly interacting with each other (Figure 2). The policy universe interacts with the policy context, which relates to the policy shape and directionality. These should be articulated, considering coherence, coordination and integration through policy mixes. Articulation refers to the need to anticipate user needs and mobilize the demand in the direction of the challenge. Coordination refers to the need to manage policies in different sectors (labor, education, industry, trade, etc.) to lead the system in the desired direction [36]. The policy shape includes the institutional, actor and policy layers at different levels. Directionality should target clear objectives and new policy tools with new metrics for assessing results. According to Kuhlmann and Rip (2018), directionality refers to the need to articulate collective priorities and the direction of change [36].

The intended outcome is a reconfiguration of sociotechnical innovation systems through new technologies’ development, new actor-network and institutional reconfiguration promoting changes in economic dynamics. The effectiveness and efficiency of transformative policies will continue to be assessed through the difference between the goals and inputs and the outputs. However, policy design can balance the equity and the efficiency objectives to reduce inequitable outcomes without blunting the economic efficiency requirement [4].

Considering the learning processes and reflexivity, there is a larger evaluation that goes beyond market failure analysis. The outcomes and societal impacts are possible to assess through learning but also through reflexivity, a way of learning how not to do it (Figure 2). Reflexibility may also refer to the ability of the systems’ agents to anticipate changes and mobilize actors [36].

Figure 2 Evaluation framework for transformative policy.

3. Discussion

Present challenges - environmental, demographic, economic or social - require a structural change that justifies accelerating the speed of innovation and its efforts through funding policies [29]. Innovation takes time but also directionality. In this way, the third generation of industrial policies proposes to guide directionality and socio-technical systems goals towards convergent problem-solution constellations [14,17,29]. Sustainable transition characteristics suppose a multi-actor interaction and participation from different institutional domains and spatial scales to achieve social policy goals. It is important to understand actors’ dynamics and power relations thus socio-technical regimes are embedded in highly complex networks, which will need to be reshaped [22,24].

According to Nilsson et al. (2021), the direction implies top-down state-driven projects and bottom-up partnerships between private and state actors to support the development of new technologies and new techno-economic paradigms [21]. Due to the complexity of multi-actor interactions and transitions nature, directionality is not a straightforward matter [15]. Trade-offs to achieve transformative change may include directionality, demand articulation, policy coordination, reflexivity and experimentation [15]. Directionality requires the setting of collective priorities that require a strategic policy approach for achieving societal transition changes through increasingly complex implementation, evaluation, and coordination processes. Policies should be directed at addressing “transformative system failures” such as lack of directionality, instead of being focused on a narrow “market failures” perspective that only stresses the economic perspective forgetting other policy domains and levels [27,45].

The ability to set a direction is a difficult challenge and requires a better definition of goals and new indicators to monitor them [11]. Missions or challenge-oriented approaches are an increase in the directionality of innovation policy fostering a sociotechnical transition [5]. Missions are about setting specific directions and deciding that a transformation must occur in society and for that purpose they must be picked or chosen strategically [29]. According to Aiginger and Rodrik (2020), mission “framework involves strategic thinking on the desired direction of travel (which road to take), the structure and capacity of public sector organizations, assessment of the way in which public policy is carried out, and the incentive structure for both the public and private sectors (risks and rewards). They emphasize the need to articulate well-defined goals or “missions” focused on solving important societal challenges.” [11]. Despite missions representing an increase in the directionality of innovation policy, some authors consider that they are a narrow perspective to fully answer the societal transformative change complexity, and TIS could have more effective results. However, new innovation processes may explore both mission and TIS characteristics to define directionality and strategy for achieving the policy objective. Despite apparently representing different policy perspectives, both policy philosophies may benefit from each other and have a common branch “policy inosculation” (Inosculation is a natural phenomenon in which trunks, branches or roots of two trees grow together). I.e. When addressing short-time or very specific challenges, missions-oriented policies may be more effective, thus, have specific goals. The involvement of a broader set of actors with diverse opinions may originate conflicts that would result in directionality failures. On the other hand, TIS’s focus on policy coordination and reflexivity may be useful to improve mission design. I.e. missions may pick the willing but poor results may emerge if these actors do not interact and co-construct knowledge with each other. TIS may be considered as a broad policy perspective, thus it requires more involvement, more actors and a wide discussion process. However, this rich diversity of perspectives may hamper the set of a direction, and the definition of a clear strategy.

The literature highlights the need to design, evaluate and monitor policies for transformational system change as well as the need for policy coherence and articulation through appropriate policy mixes to shape the directionality of socio-technical systems while avoiding transformational system failures [17]. Transformative innovation policies need an integrated evaluation of the different policy instruments and their interactions, thus failures overcome require multi-faceted policy interventions that combine multiple instruments that will need to be regularly updated to adjust directionality [23,24,45,46]. On balance, policy mix coordination and articulation among directionality, government institutions and actors, is crucial to move towards the right direction [17].

3.1 Policy Framework Application to an Energy Policy Example - Renewable Energy Communities

Not every innovation policy might be considered a transformative policy. However, in what concerns energy systems, there is a clear association between innovation and technological change and transitions. Changes in energy supply have been far-reaching since the Industrial Revolution, and oil in particular, emerged as an expensive option at the end of the 19^th century to occupy the dominant global position. Levels and structure of energy services have changed dramatically since the Industrial Revolution, reflecting population and income growth and, above all, technological change.

Without innovation systems, dominant technologies would not be exposed to competing technologies and transitions would not arise. However, innovation may be more than technology and include the re-configuration of existing technologies and actors. A possible example of the re-configuration of existing technologies and actors is renewable energy communities which have multiplied in the last years in many countries, even without favourable conditions. The research performed by [55] analyses motivations for local renewable projects in Germany and the Netherlands. The motivations were mainly associated with economic gain (such as decreasing energy costs) and normative (such as addressing climate change). But in some of these communities, other motivations were also present, such as a new experience and integration into a community. Energy community projects may exemplify how the developed evaluation framework could be applied. The sources of empirical evidence may be split into observation and experimentation, and the last method is an active phenomenon that involves intervention, such as the case of energy communities.

Considering the macro level – the policy universe – in some of the studied communities the importance of governmental support at different scales – local and regional government – was mentioned concerning both political and fund support. Grillitsch et al., (2019) mentioned the importance of the ways the state intervenes and what are the implications of these modes of engagement for society. Clearly, without government engagement, some of these communities would not feel the necessary support to advance with these projects even if they have the motivation to perform them. The engagement of governments with social goals, stimulating demand side and public demand as a driver for transformation is central. Also, De Laurentis (2012) research refers to the importance of the regional government in supporting the renewable energy industry in Wales and states that efforts need “to be done to facilitate planning control, provide skills and create new demands for renewable energy that will further foster business growth and further strengthen the existing manufacturing base and innovation in Wale” [56].

The policy context at the meso level reflects the intervention area and is shaped by multi-actor intervention. Some of these communities mentioned the benefits from the government but also the university support. Particularly at the beginning universities actively participated in the organization of the work but, after some time, the researchers became just observers and the villagers became more independent until the whole system started operating. This demonstrates that the initial knowledge support was transferred to villagers over time, and, in the final, these acquired new skills and competencies for achieving their purposes. In addition, “through the researchers, they could receive additional funds from the government, which were essential for the construction of the heating grid.”. This validates what was previously mentioned as the “quadruple helix” – the traditional support of research and innovation policies – industry, universities and government – is considered a narrow perspective that needs to be replaced by broader participation of actors, stimulating demand side and public demand as a driver of transformation, which includes not only citizens but also civil society. In the policy context variables such as time, market deployment (new technologies development), demand-side (users’ preferences), and geographic scale integration, conditionate the context. Time is relevant for technology deployment but also for people acquiring new skills, technology deployment is necessary for replacing other technologies – such as oil – and demand preferences may promote or hamper projects like renewable communities, while geographic scale integration may help to scale up some of these projects.

Within the policy shape, at the micro level, the community motivation is reflected with clear common targets. The proper choice of targets and community goals implies its discussion at the community level and the obtention of a general agreement. Most communities had economically cost-driven motivations – thus, they wanted to be independent. Some produce both heat and electricity 100% from renewable energy resources and cover all the energy needs of the community.

As previously mentioned, directionality involves more than two coordinates (x,y,z). In renewable energy communities, these variables could be technology, time, and market. Energy communities would not be possible without renewable technology, which is key for energy systems development. However, the evolution of technology, cannot be detached from time. About markets, some of these communities refer to the importance of “Being independent from big oil companies and thereby from increasing fossil prices”. The oil price was a major motivation for the establishment of renewable energy communities. In some of these communities, profit was also expected, which could be reinvested in the local community. Despite the economic and environmental benefits, this model improves people integration through debate and collaboration and engages the community “It was also fun to do it together and help each other.”.

Some of these energy communities started with top-down strategies with people wanting to start a project due to a government program for subsidizing individual applications of solar PVs. But evolved due to a bottom-up process, with different governmental scales and institutions participating towards a community goal. It is described that when the technology supplier offered a 20-30% price discount, the initiators decided to involve other people from the neighbourhood. Initially, they were not very successful, so they cooperated with other residents to organize a communication campaign to attract as many people as possible through advertisements, distributed leaflets or went to speak personally – this was the most effective strategy to engage people. This demonstrates that despite top-down governmental initiatives and government programs being relevant, bottom-up processes with individual and social society engagement may accelerate the processes. This is in accordance with Nilsson et al. (2021) perspective, in which the direction implies top-down state-driven projects and bottom-up partnerships between private and state actors to support the development of new technologies and new techno-economic paradigms [21].

3.2 Evaluation Framework Application to an Energy Policy Example - Renewable Energy Communities

The developed evaluation framework may apply to a new social reconfiguration through new competencies such as reflexibility and the constant ability to learn from both sides. Energy systems include both the supply and the demand side since one side cannot exist without the other. However, the demand side is essentially seen as the energy services that should fulfil the demand at any moment. Excluding the demand side for participating may be seen as a narrow perspective of an energy system. Developing a holistic approach that includes both sides could help to overcome some of the system failures. This is the reason why this research develops a framework that considers both the demand and supply sides and its articulation. However, the full demonstration of the framework is not possible thus there are no quantitative elements to support it – according to the framework developed, measurements and quantification have a primordial importance in assessing results.

The developed evaluation framework will require more competencies, particularly, from the demand side – society and end-consumers, thus these usually do not take part in the decision process. However, the supply side may benefit from the creation of new business models. It is not clear if mission-driven would fit into the developed framework, thus targets/objectives are very specific. If this is the case, the role of demand and supply could be neglectable, thus the chosen actors and their roles may be defined from the beginning. Despite the increasing complexity, the evaluation framework should be expected to overcome or avoid some of the identified failures. However, each case is singular, and the interpretation should be done for each case.

In most of the renewable energy communities, there was a very clear purpose: to increase independence from large fossil-fuel energy companies and decrease costs. Considering renewable energy communities, the failures that might occur in the policy universe are concerns about conflicts of interest between society’s goals and the government’s position, since earlier policy approaches give preference to centralized production or misunderstood society values interpretation. Policies captured by dominant interests may represent a potential failure towards societal goals.

Considering the failures that might occur in the policy context, these are mainly associated with the lack of market, technologies, and possibly waiting games. In what concerns renewable energy communities, these types of failures may compromise the outputs and system reconfiguration. One of the renewable energy communities’ purposes was to avoid oil price’s constant variation and diminish the risk. However, if oil prices remain low for a long time this type of initiative may be completely hampered and centralized fossil-fuel companies remain as the main actors without changes in established actors (Figure 3). Technology also plays an important role; thus, more efficient technologies have higher yields and can produce more energy with lower production costs. This will necessarily be translated into higher or lower profits for the community. The higher the profits, the higher the chances for reinvesting in equipment and services for the community’s well-being. Losses and bad economic results may demotivate the community and increase the level of conflicts.

Figure 3 Evaluation framework applied to renewable energy communities.

Concerning the policy shape, the failures are mainly associated with a lack of coordination among different actors, actors’ roles not well defined, insufficient adaptation to existing infrastructures, and institutional weakness that might compromise the different actors’ participation in the processes. In addition, directionality failures are associated with not well-defined objectives within the policy context. The renewable energy communities’ examples given are cases of success. However, due to a lack of articulation between communities and regional or national authorities, many of these projects could never take off. These communities’ developments, more than motivation need the support from a large range of actors and institutions, that accept society entrepreneurship. The coherence, coordination and integration of policies i.e. research and development at the community level and fiscal policies may promote renewable energy community projects. Thus, these communities promote social system reconfiguration, promoting new and decentralized technologies, new actors, associations and cooperatives that can share experiences with other communities leading to local economic profits and development. This agrees with Eliot D. (2000) “The successful deployment of new technology requires the existence, or the development, of suitable social and institutional contexts — a technical infrastructure, suitable financial networks, a skill base, along with the appropriate pattern of social acceptance” [57].

Despite the importance of economic earnings, it is also necessary to assess the social impact of these measures. The ability to be constantly learning from experiences and learning how to overcome obstacles maintains policies in a continuous adaptation cycle. In what concerns social system reconfiguration, energy-renewable communities are one of the best examples, thus promotes new technologies, new actors, local economic profits and development and knowledge reconfiguration.

4. Conclusion

The recent emphasis given to innovation policy reflects the growing recognition that knowledge and innovation are fundamental for economic performance and growth together with social problem mitigation. The third frame of innovation policy has a larger scope than previous innovation policies only based on the scientific and technological content of innovations, thus it accounts for other forms of learning beyond science and technology that might also lead to innovations like learning by doing, using, or interacting.

In this research, three levels of policy interactions were conceptualized. At the macro level, the policy universe reflects the governmental options that shape and define its policies. At the meso level, the policy context reflects the policy intervention area and limits the intervention area of the policy. The policy shape corresponds to the micro-level and reflects the policy-specific goals. The policy direction corresponds to the trajectory of policy shape and context within the policy universe.

Within the defined framework the failures at each level were addressed. Failures at the macro-level may not correctly define societal problems and originate other failures that compromise policy results and outcomes. However, at the meso level and micro level, more specific failures associated with directionality and coordination may emerge. To overcome these failures a more proactive and continuing evaluation of policies needs to be done.

The developed evaluation framework applies to a new social reconfiguration through new competencies such as reflexibility and the constant ability to learn from both sides. An evaluation process with more involved parts will necessarily be more complex, but the development of new social competencies with learning, reflexivity and feedback may be key to overcoming obstacles. The engagement of governments with social goals, stimulating demand side and public demand is key for transformation. When a policy framework evaluation considers only one side, it has a narrow perspective and it is not possible to evaluate the “global picture” which may induce misdealing results. An evaluation framework that considers two sides would be beneficial thus communication and continuous learning help to redefine the policy direction as it was described in the involvement of renewables communities.

Due to the importance given in the literature to learning and reflexivity, future policies should consider not only the policy outputs but also the policy impact of social outcomes with policy learning capabilities. Policy evaluation goes beyond effectiveness and efficiency, thus learning, reflexivity and accountability are considered to be decisive transformative aspects for new innovation policy accomplishment.

Author Contributions

The author did all the research work for this study.

Competing Interests

The author has declared that no competing interests exist.