This is ...?
... Building Use Studies for Licensees, for use in conjunction with BUS Methodology survey questionnaires.
We help support the public service Usable Buildings Trust website:
Introduction
This has details of the BUS Methodology occupant data reporting and analysis method.
It is intended as background for BUS licensees and others using or wanting to understand the system.
We normally use questionnaires as the basic way of gathering data. We find that this approach is the most practical and cost-effective, both for the people and managers in the buildings being studied, and for ourselves. This can be supplemented if necessary with individual telephone call follow-ups, focus groups, and revisits. We also have an internet version (q.v.).
Survey data gathered from paper questionnaires are typed into pre-formatted Excel data files. Survey data from the internet version are imported to a database. The database is then processed by a series of scripts to produce the results.
The main results are presented as an Appendix which includes:
Appendices A and B are distributed as pdf files.
You may either:
To use the BUS Methodology occupant questionnaire you must sign a licence in advance of the survey.
What is it for?
The BUS analysis method is a quick and thorough but not simplistic way of obtaining professional-level feedback data on building performance, primarily from the occupants. It may be used by itself, or with other techniques as part of a post-occupancy evaluation.
By professional-level we mean:
Who's it for?
Anyone who wants to know more about building performance. We try to give convincing answers based on trustworthy data. We are not testing hypotheses or theories (although the method can be used for this if you wish). We are not using research to promote other services. There are no hidden agendas.
The outputs are helpful to:
Who uses it?
Usage is split 60:40 between advanced design practices (along with consultancies, developers and building owners) and researchers.
The method originated in 1985, and has been used by over 200 organisations worldwide. Licensing has been in operation since 1995.
The method is either licensed by third parties or used in-house.
Most outputs are not in the public domain. A notable exception is the series of Probe post-occupancy studies, which used the BUS method under licence.
About half the buildings on the dataset are in the UK and Ireland. The rest are spread over 16 other countries.
You get with the licence
Why do we ask for a licence?
We do not make the questionnaire openly available in the first instance. We have found that asking users to sign a licence first is a better method. Our main concern is that people have been carrying out studies without either telling us or letting us incorporate their data into our benchmarks. As a result, the feedback loop was not being properly managed.
We ask for a licence:
What data do I get?
The method gives a range of quantitative and qualitative data usually including the following (depending on which version is used):
Charges
Our charging policy is normally:
We normally require that the licence fee is payable on receipt of the questionnaire before the study takes place.
Changes to the questionnaire are discouraged - because it adds extra cost - but they can be incorporated if required.
Questionnaires are best for the job
Our main analysis tool is the questionnaire. We find the questionnaire to be a good compromise in achieving our objectives. It is/gives:
Range of questionnaires now in use
The questionnaires have evolved from the original 16-page version in 1985 to a slicker 1, 2 or 3 page version now.
We normally use a paper questionnaire to collect the data because
The questionnaire is usually handed out in the morning and collected in the afternoon of the same day. We find that this gives approach us high response rates and minimises impacts on the occupants and managers. We page-set the questionnaires using InDesign software, outputting files in pdf format.
The range now covers:
All the questionnaires have a common set of ten rating scales which are used for benchmarking. We normally insist that these are retained, otherwise the benchmarking and rating system will not work properly.
Outside the basic set of questions, you can add or subtract questions, but we ask you not to change any. This is because minor changes magnify the difficulties of managing data files, especially when they are compared.
Data input
Once filled in by the respondents, the data from the questionnaires are typed in to a pre-formatted Excel file, using a standard file format with fixed variable names. The internet version sends data pre-formatted as e-mails to Building Use Studies.
We have considered scanning in the data, but have found that the volume of data does not merit batch processing. You can spend more time correcting the input for scanning as it takes to type in anyway.
Typing data in by hand also introduces a quality control phase, so that, for example, spelling can be corrected and acronyms expanded.
The survey building data files in Excel are then error and range checked using the AutoFilter feature.
We use Excel because it is cross-platform and widely available making it relatively easy for anyone in most of its versions. It is normal for our licensees to type their data using Excel. However, if you wish us to prepare the data for you then this can be carried out for a small extra fee.
Excel files are readable by FileMaker Pro and Aabel (a statistical graphics package used for publication quality output), as well as being programmable by AppleScript.
Our in-house analysis and data entry work is carried out using Apple OS X, but with software which is cross-platform so that the approach can be replicated on other platforms if necessary.
We have found that minor problems can arise when moving files from one platform to another. But they are relatively trivial compared with the advantages of transparency across platforms.
The analysis database
The data file/s from Excel are input to an in-house system using FileMaker Pro. We use FileMaker Pro because it is easy to program (especially when this is done with gaps of weeks or months between sessions) and does not require advanced technical expertise to maintain. This means that we can manage the program ourselves and make changes as we go along without it becoming too costly.
The system allows basic statistics and indicators to be calculated and output in a set format. These are provided in the Appendices to our reports.
The database can also be used to generate further analyses not covered in the basic tables.
The database approach introduces better levels of data management and considerably reduces the burden of data management across hundreds of files.
However, the system is not fully integrated and is not turnkey (i.e. it requires some specialist operation). This is deliberate. Buildings and their operating contexts are complex. Survey requirements are also constantly changing, so there will always be some fine-tuning needed. We do not think that it is practical to produce a complete 'point and go' survey instrument.
We find Excel graphics and statistics are not specific enough for our needs and there is no FileMaker Pro graphics capability, so we generate graphics and statistics to the level required using either our in-house 'slider' charts or Aabel.
Data output from the FileMaker Pro database are typed by hand into a metafile maintained in StatView to generate benchmark information, statistical tests and benchmark charts.
Our system is thus 'open'. This has the advantages of:
The FileMaker database also has a comments analysis system and a data validity checker, both in AppleScript, which organise the comments from the questionnaires and examine data ranges and type validity.
Drawbacks include:
The benchmarks datafile
This is maintained in FileMaker Pro and Aabel, a statistical and graphics program which gives us the required balance of simplicity and sophistication.
Client requirements differ appreciably so we use a 'highest common factor' approach which covers the hardest case as well as the simplest. One client might require results in a simple format ('All I want to know is whether our building is better than theirs'). The next wants statistical tests ('I need to run this past the statisticians in market research') . Others want fine detail ('Is the glare worse on the second or first floors?').
Aabel allows us to carry out most type of statistical tests quickly. It has the virtues of :
Aabel is employed for any operations on the benchmark dataset (not individual building studies). We input the results from the building surveys into the benchmark file.
The 'slider' graphics used in Appendix A show how well a study building performs against the benchmarks are created to our own specification automatically by AppleScript using data from the FileMaker Pro database. The plots use Intaglio graphics.
Why go to all this trouble? Basically, to meet the hardest case we encounter without then having to carry out major rework (which raises costs substantially).
Example of a page from the Analysis database
This is an example page from the Analysis database (one of up to 30 pages, depending on survey type). The one-page version has up to 10 pages; the 2-page version up to 21.
Normally, benchmarks are available only to licensees and are not in the public domain. This is a condition of the licence.
Example of a benchmark plot
These plots are supplied online for all main variables in the study.
The histogram shows how the responses for the variable shown are distributed amongst the sample.
The plots show how the study buildings compare with the benchmark dataset. Actual benchmarks are shown on the 'slider' graphic (ticks on top and bottom scales).
This is an example of an histogram, slider graphic and percentile plot for one of the variables from a building study. The colour of the study building data point also gives the result of the benchmark test for that variable (red is poor, amber is middling, green is good)
Can I see a copy of the questionnaire?
Yes, normally after the licence has been signed but can be before, depending on the circumstances. We will send you a copy of the questionnaire in pdf format after initial discussions with you about arrangements.
We do not make the questionnaires openly available and require that licensees do not publish them . This is not because we don't believe in open access (we do!). Our hospitality has been abused in the past and we need to introduce some level of checks and protection, especially when students are using the material.
We also need to be sure that everyone is using up-to-date versions of the questionnaires and benchmarks so that they are compatible with the latest version of our analysis software.
Components of the BUS analysis system as graphic
This has the main components of the system as a graphic showing main inputs (top and outputs (bottom).
Confidentiality
Unless you specifically instruct us otherwise, your results will be treated as strictly confidential. They will never be revealed to third parties.
If you wish to put your results into the public domain, then we may refer to them by name, as for example, in the Probe series of post-occupancy studies which use the BUS method.
As a general rule, we would encourage you to publish the results so that others may learn from them.
If your results go into the public domain by default (as with, for example, postgraduate theses) then we request that you do not to reveal the BUS benchmark numbers as a whole set (references to individual benchmarks are acceptable). This falls within the terms of the licensing agreement.
Our approach
We try to collect data to high statistical standards, and present them in an interesting and open way. We also try to simplify for those who do not need the detail. This is a delicate balance because we do not know in advance the levels of detail required.
Our results can be used:
On a technical point, we do not normalise data, except in the calculation of the Summary, Comfort and Satisfaction indices where all formulas are shown (in Appendix A).
We think that normalisation (i.e. adjusting results with weightings to try to balance out intrinsic population differences) is ultimately misleading (e.g. between buildings with different proportions of part-time staff, where greater numbers of part-time staff are known to affect ratings favourably - the less time people spend in a building, the more they say they like it!) .
We believe it is better to present raw scores, and normalise only at the final analysis stage, making it completely clear where weightings have been introduced.
It is also worth remembering that buildings in use are extraordinarily complicated, with a host of variables in contexts which are volatile. We try to capture as much of the richness and complexity as we can, without being overwhelmed by too much data.
More on the approach. This is a 156k pdf download of a presentation prepared in 2005 so some of the numbers may be out-of-date.
The basic service gives you ...
You do not normally get ...
We request you not to ...
What we would like you to do
Although we fully respect confidentiality (see Confidentiality q.v.) we also request that you consider publishing your results on the feedback website:
Here, you will find examples of other buildings studies and techniques which also may be of use.
You may also find other feedback techniques in the Feedback Portfolio:
www.usablebuildings.co.uk/fp/index.html
What's the pedigree?
The BUS occupant survey was started in 1985 as part of the first study of sick buildings in the UK - the Office Environment Survey, by Sheena Wilson and Alan Hedge (Building Use Studies, London, 1987).
Subsequently:
In all, the questionnaire has been used in over 600 buildings since its inception in 1985. The current database has 380 buildings from 17 countries.
What about questionnaires on the internet?
We use internet-based data collection and delivery, but find that the drawbacks can outweigh the advantages:
On balance, we prefer the paper method, and we think respondents do as well. It works best with samples of up to about 250 respondents. This makes the data entry manageable. See also Data Input (q.v.).
However, if clients insist on an internet-based approach we are happy to accommodate it.
What about housing?
We are developing a questionnaire for evaluating housing, which is currently under test.
Housing presents different challenges to non-domestic buildings. We do not have benchmarks for housing although plan to develop these when our samples are large enough.