Assessing Sustainability in Housing LED Urban Regeneration: Insights from a Housing Association in Northern England

Assessing Investment in Housing

Following the economic crash of 2008, it became increasingly difficult for housing associations to comply with their mission. Due to the financial shortage, prioritizing investments in the right way is more important now than in the past. The current decision-making process on investment in housing is often undertaken without having a full understanding of all the related benefits, leading to an underestimation of the overall value of the investment itself. Part of the issue is related to the lack of a robust evaluation methodology, suitable to support with an evidence-based approach the financial assessment of all the positive effects. This paper aims at filling this gap, by exploring how far a more comprehensive assessment methodology, namely the Sustainable Return On Investment (henceforth: SuROI) approach, can be useful in uncovering those hidden benefits in the specific case of the social-housing sector. Efforts to demonstrate the usefulness of this methodology have been made in sectors other than social housing. Furthermore, the existing methodology has been developed up to the stage of providing stakeholders with a robust and evidence-based overall assessment of the value of the investment as a whole. This paper discusses the specific application of SuROI to the housing sector and suggests the potential applicability of SuROI in mapping benefits and costs across the stakeholders’ community, allowing decision makers in the housing sector to negotiate the potential contributions of positively impacted beneficiaries. To do that, it first discusses the limitations of the most applied assessment methods and then introduces the basic principles of SuROI in order to pave the way for the calculation of the overall benefits of two case studies, both approached with two alternative methods, showing how SuROI allows the unveiling of benefits which are hidden in the traditional approach. The impact of this paper could be broad, since the issues highlighted above are related to a substantial and widespread problem that the findings go some way to addressing, as better clarified below.

Before the economic crash of 2008, housing associations relied heavily on bank financing to fund their development. Banks and other lenders historically provided housing associations with loans in favourable terms. Until the advent of the credit crunch, banks priced loans to housing associations at 20 to 30 basis points above the London interbank lending rate. ¹

House of Commons, Communities and Local Government Committee, Housing and the Credit Crunch. Third report of session 2008–09 (2009), accessed February 2, 2019, https://publications.parliament.uk/pa/cm200809/cmselect/cmcomloc/101/10102.htm

However, once the credit crunch came about, not only did banks charge more for new loans, but they also sought to rewrite the already existing housing association loans they had outstanding. Housing associations seeking additional financing found that banks demanded rates as high as 300 basis points above LIBOR, often more than ten times higher than their previous rates. A number of lenders refused altogether to lend to housing associations. ²

M. Hilditch, “Spot the Difference,” Inside Housing (2009); House of Commons.

Banks sharply increased the cost of credit for housing associations, whilst demand for for-sale homes and shared-ownership housing plummeted, reducing revenue for housing associations. ³

C. Dowler, “Shared Ownership Sales Slump,” Inside Housing (Aug. 28, 2009).

Housing associations had increasingly been building housing for sale in the open market to generate additional revenue to help fund the development of social housing. The crisis left housing associations with thousands of unsold housing units, whilst additionally reducing the market value of the associations’ housing stock and land holdings. Additionally, housing associations’ ability to generate funds to support the development of social housing was curtailed, whilst the crisis also reduced the amount of housing acquired from private developers through the so-called Section 106 agreements, the legal agreements between local governments and developers that usually regulate the obligations of the latter with respect to the public burdens generated by the development process. A reduced revenue subsidy for social housing, notably via housing benefit, has contributed to the financial difficulties that housing associations currently face. In addition, a reduction of welfare support will impact on the ability of tenants to pay their rent, whilst the cutting of rent paid by tenants by 1 percent for at least the next couple of years has added to the challenges faced. To cap it all, the right-to-buy scheme can force associations to sell at huge discounts. Added to this, falling wages and a lack of employment add to the issues, whilst social housing is likely to be in even greater demand, ⁴

D. Chevin, Social Hearted, Commercially Minded: A Report on Tomorrow’s Housing Associations (2013).

putting even more pressure on housing associations.

Critical Review of the Current Assessment Methodology

Following the discussion of the current financial issues challenging housing associations, this section will set the context for understanding how the assessment of the urban regeneration strategies has been typically performed so far. The aim is to appreciate the innovation behind SuROI, in comparison to recent and/or current assessment methods commonly used to pre-assess the impacts of investments on housing in different programs or initiatives. The following methods, mainly related to governmental schemes, will be reviewed: (1) EGRUP Guidance, (2) City Challenge, (3) Single Regeneration Budget, (4) New Deal for Communities, (5) Urban Development Corporations, (6) Enterprise Zones, complemented by the following methods: (7) Hemphill framework, (8) Sustainable Urban Renewal Project Assessment Model, (9) INDicator-based Impact-assessment, (10) Regeneration Balance Sheet.

The interdepartmental Evaluation Group on Regional and Urban Programmes (EGRUP) uses a framework for the ex-post evaluation of expenditure and regeneration schemes. ⁵

HM Treasury, A Framework for the Evaluation of Regeneration Projects and Programmes (London: HMSO, 1995).

The framework measures value for money and cost effectiveness and emphasizes the need to clarify objectives, identifying the rationale for engaging in specific activities and assessing their value on markets. The guidance notes that all regeneration schemes ultimately have the same aim, to promote self sustaining environmental, economic and social regeneration in areas needing intervention. The guidance focuses on economic efficiency and covers the need for outlining the inputs, outputs and outcomes of an urban regeneration scheme. Shadow pricing techniques, statistical sources and proxies are used within the evaluation, but any stakeholder participation is not targeted to an actual scheme. It is consistently stated within the evaluation document that further research is required in the area of valuation for regeneration activities for which market-based information is not available.

City Challenge ⁶

H. Russell, J. Dawson, P. Garside, and M. Parkinson, City Challenge: Interim National Evaluation (London: The Stationary Office, 1996).

was commissioned by the DETR and aimed to regenerate thirty-one areas through Regeneration Partnerships which ran in deprived urban areas between 1991 and 1998 with the aim of improving specific rundown inner-city areas and the quality of life of local residents. ⁷

DETR, Planning for Sustainable Development: Towards Better Practice (London: HMSO, 1998).

City Challenge had an emphasis on partnerships, competitive bidding for resources, multiple objectives and a multi-sector approach to regeneration. ⁸

Department of the Environment, The SRB Evaluation Framework. Discussion Article 83 (Department of Land Economy, University of Cambridge, 1997).

Each City Challenge Partnership received £37.5 million over a five-year period, depending upon the satisfactory achievement of annually agreed targets and objectives. In terms of the evaluation of this scheme, there is no one method or framework in place to cover the multitude of partnerships. Indeed, the Final Evaluation of City Challenge ⁹

DETR, Towards a Language of Sustainable Development (London: The Stationery Office, 2000).

found that there was an enormous diversity of local evaluations which translated to an insufficient level of consistency to make meaningful comparisons across all areas. A more standardized methodological approach was considered to have been of wider value. ¹⁰

DETR, Planning for Sustainable Development.

Additionally, the evaluations undertaken or commissioned by the partnerships themselves varied significantly in quality, content and approach, with some choosing to assess the effectiveness, efficiency and impact of a program, while others concentrated on updating baseline indicators. ¹¹

Ibidem.

A key aspect of the schemes supported through the City Challenge Programme was their integrated nature – with traditional housing improvements being complemented by other regeneration projects concerned, for example, with job creation, training, and crime and community safety; however, no method was set up to capture any of this information in numerical and quantifiable format. The DETR ¹²

Ibidem.

touches on the notion of sustainable development, including a statement that if a regeneration program is to be regarded as successful, the outcomes it generates must be sustainable in some form; however, no method of measuring such aspects of sustainability is listed. Along the same lines, concern was also expressed within the City Challenge Final Evaluation on the lack of monitoring requirements for the quality of outputs or outcomes.

The Single Regeneration Budget (SRB) ¹³

A. Brennan, J. Rhodes, and P. Tyler, Evaluation of the Single Regeneration Budget Challenge Fund (London: DETR, 1997).

was a single funding stream that resulted from the amalgamation of over twenty smaller regeneration-related programs. ¹⁴

D. Zuberi and A. Taylor, “Urban Renewal in Vancouver, Canada,” in The Routledge Companion to Urban Regeneration, ed. M. E. Leary and J. McCarthy (London and New York: Routledge, 2013), 292–302.

The aims of this program included bringing about multifaceted, economic and social regeneration in often relatively small neighborhood areas. ¹⁵

J. Rhodes, P. Tyler, and A. Brennan, Evaluation of the Single Regeneration Budget: Final Evaluation (Cambridge: Department of Land Economy, University of Cambridge, 2007).

The SRB evaluation methodology was designed to evaluate the process by which economic, social and physical regeneration is achieved at the local level. ¹⁶

Department of the Environment, The SRB Evaluation Framework.

A cost benefit, inputs versus outputs approach was taken in the study. Additionally, outcomes were measured in terms of indicators in such areas as jobs created or safeguarded, enhanced pupil attainment, personal development of young people and community safety initiatives. ¹⁷

Zuberi and Taylor, “Urban Renewal.”

There were three levels of outcomes involved within the evaluation. These levels cut across the distinction between social, economic and environmental goals and were termed delivery (the level of outputs), impact (the level of outcomes) and sustainability, termed as the longer-term social, economic and environmental impact over the life of a given partnership. Indicators were used to gauge economic benefits, housing benefits, social benefits, environmental benefits and community benefits, ¹⁸

Brennan et al., Evaluation of the Single Regeneration Budget Challenge Fund.

whilst an extensive baseline was also used. ¹⁹

Rhodes et al., Evaluation of the Single Regeneration Budget: Final Evaluation.

Social surveys and structured interviews were used to gauge opinions of key stakeholders within the SRB partnerships, together with an in-depth case study approach. ²⁰

Ibidem.

The social surveys included resident surveys and business surveys. However, regarding the SRB evaluation, according to some researchers there was no attempt to make even a rough evaluation of the benefits ²¹

Ibidem.

and there were no clear conclusions as to how residents benefited from a given program. ²²

S. Y. Ho, “Evaluating Urban Regeneration Programmes in Britain.” Evaluation 5(4) (1999): 422–38.

The methodology looks to cite overall net additional achievements; however, it is not able to quantify the holistic sustainable impact of the program in terms of all three aspects of the triple bottom line. In addition, it is stated that there are problems of comparability in the way the SRB is evaluated. ²³

DoE, The SRB Evaluation Framework. Discussion Article 83 (Cambridge: University of Cambridge: Department of Land Economy, 1997).

The New Deal for Communities (NDC) program, launched in 1998, was designed to help turn around the poorest of neighborhoods, ²⁴

DETR, Planning for Sustainable Development.

thus reducing the gaps between some of the poorest neighborhoods and those of the rest of the country. ²⁵

DETR, New Deal for Communities Finance Guidance (London: DETR, 2001).

The program’s ten-year horizon reflected the concern that previous Area_Based Initiatives (ABIs) had not been given enough time to instigate change. Each NDC was expected to achieve positive change in relation to six outcomes. Three of the outcomes were intended to improve the thirty-nine places. These were crime, the local community, and housing and the physical environment. Three were to improve outcomes for people: education, health, and worklessness. ²⁶

P. Lawless, “Understanding the Scale and Nature of Outcome Change in Area-Regeneration Programmes: Evidence from the New Deal for Communities Programme in England.” Environment and Planning C: Government and Policy, 29(3) (2011): 520–32.

A number of data collation and analysis tasks were central to the national evaluation, the most important of which was the biennial household survey. In 2002 a baseline was established across all thirty-nine NDC areas using a survey questionnaire. This addressed socio-demographic, status, and attitudinal considerations across all outcome areas. It was based on a random-sample survey design. Any change in NDC areas was benchmarked against other deprived, comparator areas. No previous evaluation of any English ABI at that time had been able to explore questions of net change across all relevant regeneration areas and their residents, for all outcomes, from a common baseline. ²⁷

Ibidem.

Urban Development Corporations (UDCs) were created by the 1980 Local Government and Land Act and were brought about as a political and economic mechanism for unlocking the development potential of the inner cities. There appear to be a very small number of published evaluations of the performances of the UDCs ²⁸

R. Imrie and H. Thomas, British Urban Policy: An Evaluation of the Urban Development Corporations, 2nd ed. (London: Sage, 1999).

whilst they also appeared to lack any form of systematic monitoring or evaluation. ²⁹

National Audit Office (NAO), The Achievements of the Second and Third Generation: Urban Development Corporations (London: HMSO, 1993).

CLES ³⁰

Centre for Local Economic Strategies (CLES), Inner City Regeneration: A Local Authority Perspective (Manchester: CLES, 1990).

state that evaluations involved an incoherence of figures and that there was a lack of clear and consistent information within evaluations. According to Imrie and Thomas, ³¹

Imrie and Thomas, British Urban Policy.

evaluations are performance related with a limited range of criteria, such as “jobs created and safe-guarded, hectares reclaimed and quantities of constructed roadway.” The emphasis in UDC evaluations is on value for money, with many commentators asserting that a greater range of non-quantifiable variables should have been accounted for within the scope of the evaluation. ³²

R. Hambleton and H. Thomas, eds, Urban Policy Evaluation: Challenge and Change (London: Paul Chapman Publishing, 1995).

Enterprise Zones (EZs) were first introduced into the UK in the 1980s. Their objective was to create designated economic areas which were free of tax, regulation and constraints, and where dynamic businesses could grow. After first being introduced in the 1980s, EZs have since been reintroduced by the Coalition Government, in 2011. ³³

G. Osborne, The Government Announces 11 New Enterprise Zones to Accelerate Local Growth as Part of the Plan for Growth (Press release, 2011).

The final evaluation of the original EZs was carried out in 1995 ³⁴

HM Treasury, A Framework for the Evaluation of Regeneration Projects and Programmes (London: HMSO, 1995).

and assessed the extent to which EZs had generated both additional economic activity and physical regeneration. ³⁵

J. Potter and B. Moore, “UK Enterprise Zones and the Attraction of Inward Investment,” Urban Studies 37(8) (2000): 1279–312.

The evaluation methodology covered an analysis of data, which included annual monitoring data provided by the Department of the Environment, in combination with further data collected by consultants to quantify the additional economic activity generated. Main factors analyzed included employment characteristics, the number of firms established, industrial compositions of the firms within the zones, environmental improvements and impact on the local property markets. ³⁶

HM Treasury, Framework for the Evaluation of Regeneration Projects.

Additionally, the number of jobs and costs per job created were highlighted, but the evaluation made a limited assessment of any inward investment to the zones. ³⁷

Potter and Moore, “UK Enterprise Zones.”

Face-to-face and postal surveys were sent out to local companies to gauge their perceptions of EZ benefits. Lastly, studies were carried out to assess the effects of EZ policy on local property markets, the creation of new economic activity and impact on the physical environment.

Hemphill et al.’s framework ³⁸

L. Hemphill, J. Berry, and S. McGreal, “An Indicator-Based Approach to Measuring Sustainable Urban Regeneration Performance: Part 1, Conceptual Foundations and Methodological Framework,” Urban Studies 41(4), (2004): 725–55. “Part 2, Empirical Evaluation and Case-Study Analysis,” Urban Studies 41(4) (2004): 757–72.

measures sustainability by allocating a points score to indicators within five areas including economy and work, resource use, buildings and land use, transport and mobility and community benefits. Points are allocated to each indicator and to each indicator set. Indicators are used to measure the performance of regeneration against sustainability criteria. A points scoring framework is used. However, the authors add that “although it is possible to set indicator parameters for certain regeneration outputs (number of jobs created, amount of private-sector investment levered), it is difficult to extend the same rationale to more specific and intangible sustainability criteria (quality of life, community enterprise and the social economy).” ³⁹

Ibidem, p. 726.

Additionally, although there are many sets of indicators or frameworks, there is no agreement on the application of this approach. ⁴⁰

M. K. Ng, “Reference Tools,” Social Indicators Research, 71(1-3) (2005): 441–65.

The Sustainable Urban Renewal Project Assessment Model (SURPAM) is used as a planning tool for urban design professionals and aims to assess urban regeneration schemes from a sustainability point of view. ⁴¹

K. L. G. Lee, “Sustainable Urban Model for a High Density City – Hong Kong.” Unpublished PhD Thesis. The Hong Kong Polytechnic University, 2008.

It uses weighted indicators which are chosen by the author from prior research. The indicators resulted from survey and questionnaire research with prominent architects, planners and property development managers in Hong Kong, together with some local citizens. Subsequent factor analysis was carried out. All indicators fall under the triple bottom line. The model requires input from stakeholders and can be used to properly plan a prospective urban regeneration scheme. Through usage of the model, citizens are able to express opinion on the design of a particular proposal before it becomes a reality. Scores are expressed through multiplication of the weight of a design criterion by the score indicating the performance of a scheme with respect to an individual criterion.

The INDI (INDicator-based Impact-assessment) model was formed as a result of the HQE²R project developed under the leadership of French partners CSTB and La Calade. ⁴²

C. Charlot-Valdieu and P. Outrequin, ISDIS System and INDI Model: Assessment of Neighbourhood Regeneration Scenarios with a Sustainable Development Indicators System – Methodological Presentation, HQE²R. (Sophia Antipolis: CST–La Calade, 2004).

It is a communication and decision-support tool, which visualizes the results of impact assessment of different scenarios in a neighborhood. The sustainability potential of a particular neighborhood is measured on a “sustainability scale,” with sustainability being defined by six principles (economic efficiency, social equity, environmental quality, long term, global, governance) and five global objectives: (1) To preserve and enhance heritage and conserve resources; (2) To improve the quality of the local environment; (3) To ensure diversity; (4) To improve integration; (5) To reinforce social life. The model works with sixty-three sustainable development indicators, which are linked to the five objectives above and their twenty-one more detailed targets. The tool consists of a spreadsheet, with the user inputting quantitative or qualitative values for the neighborhood for each indicator to get a resulting score. ⁴³

A. Blum and M. Grant, “Sustainable Neighbourhoods: Assessment Tools for Renovation and Development,” Journal of International Research Publications: Ecology and Safety 1 (2006): 37–54.

The remit of the Regeneration Balance Sheet ⁴⁴

C. Robbins, “The Regeneration Balance Sheet for Communities” (presentation at HQE²R work meeting No. 3 and working paper, Bristol, 2002).

is to ensure that evaluation maintains a focus on all aspects of the triple bottom line resulting from regeneration activity. The Regeneration Balance Sheet can be used either as a checklist, with ticks being placed in boxes to confirm that a particular piece of information is available to make an evaluation, or alternatively as an observation sheet regarding baseline information, objectives, indicators, outcomes or direction of change. The Balance Sheet is a very simple method of assessing sustainability impact within a regeneration scheme and can be classed as a tool for providing a structure to an evaluation and giving an overview of all its elements. ⁴⁵

Blum and Grant, “Sustainable Neighbourhoods.”

Many authors note the usage of ad-hoc “check-lists” of sustainability without a clear methodological framework. ⁴⁶

H. Barton, ed.. Sustainable Communities: The Potential for Eco-Neighborhoods (London: Routledge, 2000); H. Barton, M. Grant, and R. Guise, Shaping Neighbourhoods: A Guide for Health, Sustainability and Vitality (London: Spon Press, 2013); S. Bell and S. Morse, Measuring Sustainability. Learning from Doing (London: Earthscan, 2003); D. A. Brownhill, Sustainability Checklist for Developments (Watford: British Research Establishment, 2002).

Towards a More Comprehensive Appraisal of the Impacts from Housing Investment: Implementing the Sustainable Return on Investment (SuROI) Approach

The review of assessment methods and tools discussed in the previous section allows the assertion that, despite there being previous tools within the field of urban regeneration which have evaluated urban regeneration schemes, there appear to be none which take into consideration the hidden social and environmental benefits of a scheme. In addition, no tool then goes on to quantitatively measure such hidden benefits by the usage of “Sustainable Return on Investment” (SuROI). ⁴⁷

E. Bichard, “Developing an Approach to Sustainable Return on Investment in the UK, Brazil and the USA: A Report for the Royal Institution of Chartered Surveyors Research Trust,” 2015.

Sustainable Return on Investment is the method by which the assessment of hidden social and environmental benefits can be carried out. It incorporates such frameworks as Social Return on Investment (SROI) or Ecosystem Services Analysis (ESA). According to Nicholls et al., ⁴⁸

J. Nicholls, E. Lawlor, E. Neizert and T. A. Goodspeed, A Guide to Social Return on Investment (Liverpool: SROI Network, 2012), 8.

“things which get bought and sold take on greater significance.” Sustainable Return on Investment takes this mantra on board ⁴⁹

Bichard, “Sustainable Return on Investment.”

and places a numerical value on the social and environmental components of the triple bottom line: aspects which are normally difficult to compare against more easily measured economic impacts. ⁵⁰

S. Conejos, C. Langston, and J. Smith, J. “AdaptSTAR Model: A Climate-Friendly Strategy to Promote Built Environment Sustainability,” Habitat International 37 (2013): 95–103.

Other authors cite the typical difficulty in measuring the social “pillar” due to its abstract nature and consider that it could be overshadowed by the economic and environmental domains. ⁵¹

M. Lehtonen, “The Environmental-Socio Interface of Sustainable Development: Capabilities, Social Capital, Institutions,” Ecological Economics 49 (2004): 199–214; M. Davidson, “Social Sustainability: A Potential for Politics?” Local Environment 14 (2009): 607–19; B. Littig and E. Griessler, Social Sustainability: A Catchword Between Political Pragmatism and Social Theory,” International Journal for Sustainable Development 8 (2005): 65–79.

Xing et al. ⁵²

Y. Xing, R. Horner, M. El-Haram, J. Bebbington, “A Framework Model for Assessing Sustainability Impacts of Urban Development,” Accounting Forum 33 (2009): 209–24, p. 210.

cite that one of the main challenges can be a difficulty in the measurement of what they term “apples and pears,” which is to compare the measurement of costs and values, which are expressed in different units. The SuROI method potentially solves this issue. In addition, recent guidance from the Royal Institution of Chartered Surveyors ⁵³

RICS. RICS Valuation—Professional Standards (London: Royal Institution of Chartered Surveyors, 2014).

recognizes the need to include a wider range of factors that can influence the value of built environment projects and asserts that sustainability considerations are now considered as being important when undertaking valuations. ⁵⁴

Bichard, “Sustainable Return on Investment.”

The Social Value Act (2012) additionally requires that economic, environmental and social benefits are taken into consideration as part of any procurement processes, showing that the focus on sustainability is perhaps starting to change and become more important at national level also.

SuROI aims to allow the environmental and social value of a project, program or policy in the built environment to be made explicit through evidence and be added to capital costs to give an overall sustainable value. ⁵⁵

Bichard, “Sustainable Return on Investment.”

SROI compares the prospective social benefits of a particular scheme against its costs and ESA takes the costs and benefits of the environment into consideration. ESA covers both the natural and built environment, including architectural aspects within its definition. ⁵⁶

Department for Environment, Food and Rural Affairs. An Introductory Guide to Valuing Ecosystem Services (London: DEFRA, 2007).

Research Methodology

To demonstrate how far SuROI can support a more comprehensive evaluation of the benefits deriving from investment in housing schemes, two extant case studies have been selected, drawing from a housing association based in north-west England. City West Housing Trust (CWHT) is a not-for-profit housing association, based in West Salford that owns and manages over 14,600 homes in the north-west of England. It launched in October 2008, following a stock transfer from Salford City Council. Almost £250 million has already been invested in homes and neighborhoods, and further major projects are underway. However, these projects work against a backdrop of difficult economic times. The two case studies to be covered within this paper have been chosen due to the range of housing types on offer, the amount of community engagement carried out with stakeholders by CWHT and the potential for future research. Both cases have been investigated by analyzing internal reports and documents and by interviewing selected key informants from CWHT.

The first case study is a high-rise scheme. CWHT has invested £43.2M to improve 666 high-rise flats across nine blocks in Eccles, Salford. Improvements include approved thermal cladding, enclosed conservatory balconies, self-cleaning windows and new lifts, whilst internal improvements include remodelling to provide an open-plan living space as well as new kitchens, bathrooms, security doors and heating and ventilation systems. The figures below show before and after photos of the high-rise blocks.

The second case study is an environmental scheme. During 2014/15 CWHT delivered high specification environmental improvements to 476 customer homes. These improved the physical appearance of neighborhoods, enhanced property security and provided off-street parking. The figures below show photos of before and after the works.

Case Studies Development

The empirical evidence has been collected around the two case studies, as described in the previous section. In order to appreciate the contribution of SuROI to a better understanding of the benefits achievable through investment in housing, both case studies have been assessed twice, by adopting both a “traditional” approach and the SuROI approach, allowing a comparison of the numerical values from the two methods. As the “traditional” method, the authors decided to select the Single Regeneration Budget (SRB), ⁵⁷

F. Robinson, T. Blackman, and C. Stephens, Monitoring and Evaluating City Challenge, Research for Policy 1995 (Newcastle, UK: Local Authorities Research and Intelligence Association, 1995), 135–45.

because SBR Budget projects were subject to a considerable amount of monitoring and evaluation, and the review of the SRB conducted from the Department of Land Economy at Cambridge University ⁵⁸

J. Rhodes, P. Tyler, A. Brennan, S. Stevens, C. Warnock, and M. Otero-Garcia, Lessons and Evaluation Evidence from Ten Single Regeneration Budget Case Studies – Mid Term Report (London: DETR, 2002).

had been referred to as the most extensive piece of evaluation work on the impact of urban regeneration interventions. Indeed, the SRB application spans quite a long period, running from 1994 until 2004 as part of a package of measures to make Government activity more responsible to local needs and priorities in England. There was also an encouragement of mainstream service deliverers to focus more resources on deprived areas. It was additionally more flexible than previous Government-led initiatives, in that it could vary according to size and geography. A hands-off management approach was also a common feature, with local partnerships being responsible for the management of their own regeneration schemes. ⁵⁹

Ibidem.

There is a great deal of information available regarding the evaluation of the many SRB schemes carried out, including many reports, case studies and annexes. Rhodes et al. ⁶⁰

Ibidem.

assert that one feature of a good evaluation framework is that it should enable the outputs or benefits of a policy or program to be compared with the inputs or costs, stating that it is only with information of this kind that an assessment of value for money can be made. Reference is also made to existing methodology being “too narrow, too heavily output based and full of estimation problems.” It cites the issue of qualitative changes associated with regeneration outcomes, which can be given a quantitative dimension by “using sample survey techniques by indicating the extent to which perceptions in the resident population have changed.” ⁶¹

Rhodes et al. “New Developments in Area Based Initiatives in England: The Experience of the Single Regeneration Budget,” Urban Studies 40(8) (2003): 1399–426.

This is the exact technique used within our version of this study below.

However, in terms of a straightforward way of going about the evaluation process, the “suggested approach” provided by Rhodes et al. ⁶²

Ibidem.

in paragraph A.1.2 of the full evaluation report’s annexes will be used. This approach is as follows:

1. Obtain information on the activities and gross outputs associated with the SRB projects and the expenditure incurred disaggregated by key funding source;

2. Assess the additionality of SRB projects using five categories:

   	Range (%)	Mid-point
   Negligible additionality	0–9	4.5
   Low additionality	10–34	22.0
   Medium additionality	35–54	44.5
   High additionality	55–74	64.5
   Very high additionality	75–100	87.5

3. Other information will need to be assembled for the relevant projects relating to intended beneficiaries, displacement, substitution and leakage effects;

4. Assess the additionality of the public-sector financial contribution to the funding of the project. This will require an analysis of mainstream bending according to whether it has taken place over short or relatively long distances;

5. Establish the baseline position in relation to key outcome indicators in each SRB area drawing upon a range of published and unpublished data sources and through the use of social surveys;

6. Establish evidence for each of the key outcome indicators in the SRB area at specific times during the life of the SRB and identify gross changes that lie behind the net outcomes. (The social surveys are essential in order to establish the gross changes that lie behind changes in net outcomes);

7. Combine evidence from project outputs with evidence on changes in area key outcomes, remembering that there are difficult measurement problems, that change takes place often only relatively slowly, and people move into and out of the areas that are the subject of the SRB assistance. This has clear implications for the scale of the survey work that should be undertaken.

The calculations undertaken on the high-rise scheme investment according to the SRB methodology and the SuROI approach show that by following the former methodology, the Net Present Value of impacts is £2,853,144 whilst by following the latter, SuROI approach, the total is £24,101,336.08. This implies a difference of £21,248,192.08, which represents the “invisible” benefit to society that SuROI allows the unveiling of, in comparison with the SRB. Likewise, the undertaking of similar calculations on the environmental-led scheme investment enables the unveiling of an extra Net Present Value of £2,183,060.79, stemming from the difference between the Net Present Value of the environmental scheme investment calculated following the application of SuROI, minus the Net Present Value of the same scheme calculated following the SRB methodology.

Discussion and Conclusions

Some observations can be gleaned from the comparison between the SRB evaluation and the SuROI approach. Within the SRB, there would appear to be a lot less information to hand from which to make strategic decisions or conclusions on the level of impact of a given scheme. In comparison, because the SuROI method in effect “makes the invisible visible,” previously intangible areas become tangible, meaning that more information is readily available to decision makers, which in theory will increase the accuracy of decision making in the field of housing-led urban regeneration. Additionally, not all impacts of a scheme are going to be quantitative. The SRB-led methodology quantifies qualitative values in terms of the change in numbers of respondents or percentage change of response. This provides accurate information from directly involved stakeholders, but this perhaps does not bring into play sufficient supporting data. By utilizing the SuROI method, further indicator sets and proxies, brought about by tapping into sometimes thousands of responses again provide much more information, which in theory increases the accuracy of the impact and the accuracy of any subsequent strategic decision making.

It can be difficult via the traditional method of evaluation, to come to a conclusive decision on actually what the clear-cut, holistic and sustainable impact of the scheme is. There is no clear-cut, quantifiable and easy to understand ratio depiction of the impact or change promoted by a scheme, whereas within SuROI this is a fundamental part of the conclusion to the process. The traditional evaluation additionally struggles to adequately compare different forms of figures available. What is the value of one percentage point or one person’s response within the evaluation? Is it the same or different from £1 of return? SuROI makes it easy to contrast figures and results, by quantifying them all in the common unit of a monetary value. Additionally, SuROI can use many different tools within its framework to gauge various impacts from social benefits to well-being, to environmental benefits. This makes SuROI a highly flexible and integrated approach. This level of flexibility and integration does not appear within the traditional evaluation. To ensure consistency in approach, SuROI’s following of the Social Value SROI guidance principles brings about a discipline to an evaluation that does not appear to be present with the traditional evaluation approach involved with the SRB method. Lastly, with the SRB methodology, it would not be possible to predict the effects of a housing-led urban regeneration scheme which is planned for the future. With SuROI, because of the amount of data the method taps into, an accurate assessment could be gauged to again help strategic decision making.

In conclusion, in this paper we have highlighted that a gap exists in terms of evaluation methodology because current approaches do not highlight and make explicit previously intangible and hidden social and environmental impacts from housing-led urban regeneration schemes. The discussion and observations through the comparison of a traditional evaluation approach modeled on the Single Regeneration Budget program which ran from 1994 to 2004, with the evaluation method put forward as a viable alternative, that of Sustainable Return on Investment (SuROI), show that room for significant improvements in assessment methods still exists.

While environmental and social spillovers are largely disregarded because of a gap in the evaluation tools, implementing a more holistic and comprehensive methodology such as SuROI in the housing sector can be extremely beneficial in being able to capture those hidden benefits.

Finally, the assessment methodology can be easily adaptable by being refocused in terms of stakeholders’ engagement. By repacking the financial calculation, allocating costs and benefits across the different stakeholders involved, it would be possible to attract potential new investors, willing to increase the benefits that the method has unveiled. A thorough exploitation exceeds the scope of this paper and will be discussed in depth in future development of this research.