Notices
Welcome to Analyse-it

Part I. Administrator's Guide
Chapter 1. Installation

Chapter 2. Licensing

Chapter 3. Software updates

Chapter 4. Troubleshooting

Part II. User's Guide
Chapter 5. Getting to know Analyse-it

Chapter 6. Preparing data for analysis

Chapter 7. Working with analyses

Chapter 8. Feedback and crash reports

Part III. Statistical Reference Guide
Chapter 9. Distribution

Chapter 10. Compare groups

Chapter 11. Compare pairs

Chapter 12. Contingency tables

Chapter 13. Correlation and association

Chapter 14. Principal component analysis (PCA)

Chapter 15. Factor analysis (FA)

Chapter 16. Item reliability

Chapter 17. Fit model

Chapter 18. Method comparison / Agreement

Chapter 19. Measurement systems analysis (MSA)

Chapter 20. Reference interval

Chapter 21. Diagnostic performance

Chapter 22. Survival/Reliability

Chapter 23. Control charts

Chapter 24. Process capability

Chapter 25. Pareto analysis

End-User License Agreement (EULA)
Bibliography

Notices

© 2023 Analyse-it Software, Ltd. All rights reserved.

Under copyright laws, no part of this manual may be copied or reproduced without the prior written consent of Analyse-it Software, Ltd.

The End User License Agreement (EULA) governs your rights to use the software.

Analyse-it and Analyze-it are registered trademarks of Analyse-it Software, Ltd. Use of the Analyse-it logo for commercial purposes without written consent of Analyse-it Software, Ltd. may constitute trademark infringement.

Microsoft and Excel are trademarks or registered trademarks of Microsoft Corporation, Inc. Other company product names mentioned herein are trademarks of their respective companies.

Every effort has been made to ensure that the information in this manual is correct. However, Analyse-it Software, Ltd. is not responsible for any errors or omissions.

- Analyse-it Software, Ltd.

- The Tannery

- 91 Kirkstall Road

- LEEDS LS3 1HS

- United Kingdom

Welcome to Analyse-it

Analyse-it brings powerful statistical analysis and data visualization into Microsoft Excel.

This guide provides information to help you understand important concepts and instructions to accomplish tasks in Analyse-it. In addition to this book, other resources are available to help you:

- Tutorials let you quickly get started using the main features of Analyse-it with real-world examples. To view tutorials anytime, open Analyse-it and on the Analyse-it ribbon tab, choose Analyse-it > Tutorials.

- On-screen help contains all the information in this book at your fingertips. To open on-screen help, open Analyse-it and on the Analyse-it ribbon tab, choose Analyse-it > Help.

- Context-sensitive help provides brief text descriptions for most on-screen items. To see a help tag, hover the mouse pointer over a user interface item for a few seconds.

- Our support website provides all the latest information www.analyse-it.com/support.

Part I. Administrator's Guide

Chapter 1. Installation

You must install the software on your computer before you can use it.

- System requirements
Hardware and software requirements to run Analyse-it.

- Installing the software
Install the software for a per-user license or free trial.

- Installing the software for concurrent use
Install the software for a concurrent-user license.

- Transferring the software to another computer
You can move the software to another computer when you get a new computer.

- Uninstalling the software
Uninstall the software to remove it from your computer.

System requirements

Hardware and software requirements to run Analyse-it.

Microsoft Windows Vista, Windows 7, Windows 8, Windows 10, or later.

Microsoft Excel 2007, Excel 2010, Excel 2013, Excel 2016, Excel 2019, or Excel 365 (desktop installed editions).

Internet connection to enable automatic activation, software update checks, and feedback services. If your computer is behind a firewall, ensure that TCP port 443 (HTTPS) on the firewall is open for communication to secure.analyse-it.com.

Installing the software

Install the software for a per-user license or free trial.

- Ensure that your computer meets the system requirements.

- Ensure that you have the necessary administrative rights to install the software.

1. To install the software:

- From a downloaded .EXE file, double-click the file you downloaded.

- From a CD-ROM, insert the CD into the CD-ROM drive. The installer should automatically start, but if not, click Start > Run, and then type: D:\SETUP.EXE (where D: is the drive letter of your CD-ROM drive).

The Setup window opens.

2. Click Next.
The installer checks your computer meets the system requirements, and if necessary installs Microsoft .NET 4.0.

3. Read the End-User License Agreement (EULA) and then:

- If you agree, select I accept the terms of the License Agreement, and then click Next.

- If you have any questions, contact support@analyse-it.com. Click Cancel to abort the installation.

4. Optional: If you want to change the installation folder, click Change and select the folder.

5. Click Next.

6. Click Install.
Wait while the software installs on your computer.

7. Click Start Analyse-it....

Analyse-it automatically starts whenever you start Microsoft Excel.

To use the software, you must activate a license.

Related tasks

Activating a license

Related reference

System requirements

Installing the software for concurrent use

Install the software for a concurrent-user license.

Note: Installation for concurrent-user licenses is different from versions earlier than 3.00. The software is now installed and activated on each computer, and your internet connection is used to manage concurrent use. Earlier versions installed the software on a network share and used local license control. This new method reduces network traffic, allows installation onto laptops and computers, not on the same network, and depending on policy allows end-users to install the software without systems administrator assistance.

- Ensure that your computer meets the system requirements.

- Ensure that you have the necessary administrative rights to install the software.

- Ensure that TCP port 443 (HTTPS) on the firewall is open for communication to secure.analyse-it.com on each computer. Analyse-it uses your internet connection to communicate with our license server to manage the concurrent use of the software.

1. To install the software:

- From a downloaded .EXE file, double-click the file you downloaded.

- From a CD-ROM, insert the CD into the CD-ROM drive. The installer should automatically start, but if not, click Start > Run, and then type: D:\SETUP.EXE (where D: is the drive letter of your CD-ROM drive).

The Setup window opens.

2. Click Next.
The installer checks your computer meets the system requirements, and if necessary installs Microsoft .NET 4.0.

3. Read the End-User License Agreement (EULA) and then:

- If you agree, select I accept the terms of the License Agreement, and then click Next.

- If you have any questions, contact support@analyse-it.com. Click Cancel to abort the installation.

4. Optional: If you want to change the installation folder, click Change and select the folder.

5. Click Next.

6. Click Install.
Wait while the software installs on your computer.

7. Click Close.

8. Repeat installation on each computer where you want to use the software. There is no limit to the number of installations as our license server manages the concurrent use of the software.

Analyse-it automatically starts whenever you start Microsoft Excel, but a license is only acquired when you use a command on the Analyse-it ribbon tab.

To use the software, you must activate a license.

Related tasks

Activating a license

Related reference

System requirements

Transferring the software to another computer

You can move the software to another computer when you get a new computer.

It is important before you move the software that you deactivate the license on your existing computer otherwise you may be unable to activate the license on your new computer.

1. Deactivate the license on your existing computer.

2. Install the software on your new computer.

3. Activate the license on your new computer.

Uninstalling the software

Uninstall the software to remove it from your computer.

Note: You do not need to uninstall the software to install an update.

1. Start the Windows Control Panel.

2. Open the list of installed programs:

- On Windows XP, double-click Add/Remove Programs

- On Windows Vista or Windows 7, and later, click Programs, and then click Programs and Features.

3. In the list of installed applications, select Analyse-it for Microsoft Excel, and then click Uninstall.
Wait while the software uninstalls from your computer.

Chapter 2. Licensing

License activation helps you meet the terms of the software license agreement and reduces piracy ensuring continued development of the software.

When you request a 30-day trial or purchase a license to use the software you receive a product key. The product key identifies features of the software you can use, the length of time you can use it for, and how many users can use it.

For all products we offer either:

- A perpetual license that does not expire and includes 1-year of software maintenance, so you receive all software updates within the year. You can renew maintenance to continue receiving software updates, but it is not mandatory. If you do not renew maintenance you can continue using the software without further cost, although you only have access up to the last version released within your maintenance period.

- An annual license that expires 1-year after the initial activation and includes software maintenance, so you receive all software updates within the year. After expiry, you must renew the license to continue using the software.

There are two types of license seats:

- Fixed, per-user licenses allow a fixed number of users to install and use the software on their computer, and optionally on their laptop computer for use when out of the office. The number per-user licenses determine the number of users who can install the software. For example, a 2-user license allows two users to install and use the software. The user can transfer the license to another computer when replacing computers.

- Floating, concurrent-user licenses allow an unlimited number of users to install the software on their computer or laptop computer, but limits the maximum number of users of the software at any time. For example, a 5 concurrent-user license allows any number of users to install the software, but only allows up to 5 users to use it at the same time. All computers must have internet access so the software can manage concurrent use.

When you activate a license on your computer the product key and a site identifier are sent to our activation server. The site identifier is a non-unique string generated using the computer's hardware configuration. In the request, we also include diagnostic information such as the version of Microsoft Windows and Microsoft Excel you are using, plus some performance metrics of your computer. We use the information to shape the future development of Analyse-it. We do not collect any personal information except the name, organization and e-mail address you enter during activation.

Related concepts

Maintenance

Related tasks

Activating a license

Deactivating a license

Acquiring a concurrent license

Releasing a concurrent license

Finding out who's using a concurrent license

Activating a license

Activate a license to use the software on your computer.

- Ensure that TCP port 443 (HTTPS) on the firewall is open for communication to secure.analyse-it.com on your computer. Analyse-it uses your internet connection to simplify activation.

1. Start Microsoft Excel to start Analyse-it.

2. On the Analyse-it ribbon tab, click Analyse-it, and then click Activate.
The Activate window opens.

3. Click Next.
The Activate window shows the license details.

4. Enter the Product Key and the Name, Organisation, and E-mail of the registered user of the software.
The registered user may occasionally receive an e-mail with important information about updates to Analyse-it. They can opt-out via the unsubscribe link in the e-mail they receive.

5. Click Next.
The software uses your internet connection to verify the product key and activates the license on your computer.

6. If activation fails for any reason you must activate manually:

1. Click Get Activation Code.
Your browser opens the manual activation page at http://analyse-it.com/activate. If your computer does not have an internet connection, you can open the page on any computer or device with an internet connection.

2. On the manual activation page, enter the Product Key and Site ID from the Activate window.

Note: The Site ID is unique to each computer. Do not contact technical support asking for a Site ID as we cannot provide it. It only shows on the Activate window in the software.

3. Enter the Name, Organisation, and E-mail of the registered user of the software.

4. Click Get Activation Code.
The activation code shows in your browser window.

5. In the Activate window, enter the Activation code from your browser window.

6. Click Next.

7. Click Close.

Related concepts

Licensing

Related reference

End-User License Agreement (EULA)

Deactivating a license

Deactivate the license before you move the software to a different computer.

- Ensure that TCP port 443 (HTTPS) on the firewall is open for communication to secure.analyse-it.com on your computer. Analyse-it uses your internet connection to simplify deactivation.

1. Start Microsoft Excel to start Analyse-it.

2. On the Analyse-it ribbon tab, click Analyse-it, and then click Deactivate.
The Deactivate window opens.

3. Click Next.
The software uses your internet connection to deactivate the license.

4. If deactivation fails for any reason you must manually deactivate:

1. Click Deactivate to manually deactivate the license.

Your browser opens the manual deactivation page at http://analyse-it.com/deactivate. If your computer does not have an internet connection, you can open the page from any computer or device with an internet connection to complete the deactivation of the license.

2. On the manual deactivation page, enter the Product Key and Site ID from the Deactivate window.

Note: The Site ID is unique to each computer. Do not contact technical support asking for a Site ID as we cannot provide it. It only shows on the Deactivate window in the software.

3. In the Deactivate window, Click Next.

5. Click Close.

You can no longer use the software on your computer.

Acquiring a concurrent license

Acquire a concurrent-use license to use the software.

The software acquires a license when you use a command on the Analyse-it ribbon tab, so you do not normally need to acquire a license manually.

- Ensure that you have activated the software with a concurrent-user license.

- Ensure that TCP port 443 (HTTPS) on the firewall is open for communication to secure.analyse-it.com on each computer. Analyse-it uses your internet connection to communicate with our license server to manage the concurrent use of the software.

1. Start Microsoft Excel to start Analyse-it.

2. On the Analyse-it ribbon tab, click Analyse-it, and then click Acquire.
The software connects to our license server and acquires a license so you can use the software.

If a license is acquired, you can now use the software. If all licenses are currently in use, you can try again later, can ask others using the software to release the license if they are no longer using the software, or you may consider purchasing more concurrent-user licenses to avoid the problem in future.

Releasing a concurrent license

Release a concurrent-use license so others can use the software.

The software releases the license when you close Microsoft Excel, but you can immediately release the license allocated to you if you want to continue using Excel but want to let another user start using Analyse-it.

- Ensure that you have activated the software with a concurrent-user license.

- Ensure that TCP port 443 (HTTPS) on the firewall is open for communication to secure.analyse-it.com on each computer. Analyse-it uses your internet connection to communicate with our license server to manage the concurrent use of the software.

1. Start Microsoft Excel to start Analyse-it.

2. On the Analyse-it ribbon tab, click Analyse-it, and then click Release.
The software connects to our license server and releases the license allocated to you.

Another user can now use the software. If you subsequently use a command on the Analyse-it ribbon tab, a license is acquired for you again.

Finding out who's using a concurrent license

Find out who is using a concurrent license so that you can manage licenses.

- Ensure that you have activated the software with a concurrent-user license.

- Ensure that TCP port 443 (HTTPS) on the firewall is open for communication to secure.analyse-it.com on each computer. Analyse-it uses your internet connection to communicate with our license server to manage the concurrent use of the software.

1. Start Microsoft Excel to start Analyse-it.

2. On the Analyse-it ribbon tab, click Analyse-it, and then click About Analyse-it.
The Active user's list shows who is currently using the software.

3. Click Close.

Related concepts

Licensing

Chapter 3. Software updates

Software updates keep your software up-to-date with the latest features, bug fixes, and keeps it working when Microsoft release a new version of Microsoft Excel.

We regularly make updates to the software:

- Major updates include new features and important bug fixes.

- Minor updates address bug fixes that only affect a few customers or special situations.

The software periodically uses your internet connection to check for updates and notifies you if an update to the software is available. We do not notify you of minor updates that affect only a few customers; we only notify customers affected directly by e-mail or through the crash reporting system built-in to the software instead.

You must have active maintenance to download and install updates to the software.

Related concepts

Maintenance

Related tasks

Updating the software

Maintenance

Maintenance provides you with access to all software updates released during your maintenance period.

If you purchase a perpetual license, the initial purchase includes 1-year of maintenance. Renewing maintenance is optional. We notify you by e-mail 30 days before maintenance on your licenses expires and again on the day it expires. If maintenance expires you can continue to use the software but you do not receive any updates released after the maintenance expiry date.

If you purchase an annual license, maintenance is included and terminates when the license expires. You must renew the license to continue using the software.

Related concepts

Software updates

Related tasks

Checking maintenance expiry and renewing

Updating the software

Check that the software is up-to-date and includes all the latest features and bug fixes (including major and minor fixes).

- Ensure that you have an active maintenance.

- Ensure that TCP port 443 (HTTPS) on the firewall is open for communication to secure.analyse-it.com on your computer. Analyse-it uses your internet connection to check for updates.

1. Start Microsoft Excel to start Analyse-it.

2. On the Analyse-it ribbon tab, click Analyse-it, and then click Check for Updates.
The Software Update window opens.

3. Click Check Now.
The software uses your internet connection to check for updates.

4. If updates are available:

- Click Download and Install to download and install the update.

- Click Download to download the update for installation later.

When installing an update do not uninstall the existing version. If you do, the license is deactivated, and you must activate the license again before you can use the software.

5. Click Close.

Related concepts

Software updates

Checking maintenance expiry and renewing

Maintenance provides you with access to software updates.

- Ensure that you have activated the software.

1. Start Microsoft Excel to start Analyse-it.

2. On the Analyse-it ribbon tab, click Analyse-it, and then click Extend Maintenance.
Your browser opens the renew maintenance page at http://analyse-it.com/store/maintenance where you can see the maintenance expiry date and the cost to renew maintenance.

Related concepts

Maintenance

Chapter 4. Troubleshooting

If Analyse-it does not start when you start Microsoft Excel, you should troubleshoot the installation.

You can determine if Analyse-it is running by looking for the Analyse-it tab in the Excel ribbon. If you cannot see the tab you should follow these steps:

1. Checking the version installed
Check you are running Analyse-it version 3.00 or later.

2. Enabling the add-in
Ensure the Analyse-it add-in is active in Microsoft Excel.

3. Repairing the installation
Ensure that all program files are properly installed and registered.

4. Contacting technical support
Contact us if you have any problems or questions.

Checking the version installed

Check you are running Analyse-it version 3.00 or later.

1. Start the Windows Control Panel.

2. Open the list of installed programs:

- On Windows XP, double-click Add/Remove Programs

- On Windows Vista or Windows 7 and later, click Programs, and then click Programs and Features.

3. Check Analyse-it for Microsoft Excel appears in the list of applications and that the version number is 3.00 or later.
If the version is less than 3.00, or you cannot find Analyse-it in the list, install the latest version.

Previous topic: Troubleshooting

Next topic: Enabling the add-in

Enabling the add-in

Ensure the Analyse-it add-in is active in Microsoft Excel.

Microsoft Excel disables an add-in if it causes frequent crashes. This behavior prevents an add-in crashing Microsoft Excel so often that you are unable to use it. Sometimes these crashes are caused by interactions between incompatible, or unreliable add-ins installed on your computer.

1. Start Microsoft Excel.

2. Open the Microsoft Excel Options window:

- On Excel 2007, click the Office button, and then click Excel Options.

- On Excel 2010 and Excel 2013, click File, and then click Options.

The Excel Options window opens.

3. Click Add-Ins.
The list of installed add-ins opens.

4. If Analyse-it appears in the Disabled Application Add-ins section of the Add-ins list:

1. In the Manage drop-down list, select Disabled Items, and then click Go....
The Disabled Items window opens.

2. Select Add-in: Analyse-it for Microsoft Excel.

3. Click Enable and then click Close.
The COM Add-Ins window closes.

4. Click OK.

5. Exit and restart Microsoft Excel.
Analyse-it should automatically start.

Previous topic: Checking the version installed

Next topic: Repairing the installation

Repairing the installation

Ensure that all program files are properly installed and registered.

- Ensure that you have the necessary administrative rights to install software on the computer.

1. Start the Windows Control Panel.

2. Open the list of installed programs:

- On Windows XP, double-click Add/Remove Programs.

- On Windows Vista or Windows 7 and later, click Programs, and then click Programs and Features.

3. In the list of installed programs, select Analyse-it for Microsoft Excel.

4. Click Repair.
The installer repairs the installation to ensure all program files are properly installed and registered.

5. Start Microsoft Excel.
Analyse-it should automatically start.

Previous topic: Enabling the add-in

Next topic: Contacting technical support

Contacting technical support

Contact us if you have any problems or questions.

E-mail support@analyse-it.com.

Previous topic: Repairing the installation

Next topic: Getting to know Analyse-it

Part II. User's Guide

Chapter 5. Getting to know Analyse-it

Analyse-it provides an intuitive user interface, built on the familiar foundation of Microsoft Excel. Its logical task-based workflow makes sense to those that aren't programmers or full-time statisticians.

All the commands are available on the context-sensitive ribbon tab labeled Analyse-it.

Dataset context

When you are on a standard Excel worksheet the Analyse-it ribbon tab shows commands to manage datasets and create new statistical analyses. The dataset task pane shows the dataset and variables properties.

1. Analyse-it ribbon tab.

2. Dataset commands.

3. Statistical analyses commands.

4. Dataset task pane.

5. Analyse-it command.

Analysis context

When you switch to an analysis report the Analyse-it ribbon tab shows commands to manage the report and perform additional statistical tasks. The analysis task pane shows options for the statistical analysis grouped in panels that facilitate a logical workflow.

1. Analyse-it ribbon tab.

2. Report commands.

3. Analysis task pane.

4. Task panels.

5. Add analysis task commands.

6. Remove analysis tasks.

Chapter 6. Preparing data for analysis

Analyse-it takes your data for analysis directly from an Excel worksheet. It also provides commands to arrange and structure your data ready to analyze. In addition, you can use the built-in Excel commands to transform, sort, and filter data.

- Datasets
A dataset is a range of contiguous cells on an Excel worksheet containing data to analyze.

- Variables
A variable is an attribute of an object. The value of a variable can vary from one thing to another.

- Measurement scales
Five different scales are used to classify measurements based on how much information each measurement conveys.

- Missing values
Missing values occur when no data is recorded for an observation; you intended to make an observation but did not. Missing data are a common occurrence and can have a significant effect on the statistical analysis.

- Case and frequency form data
Most data are available in case form with data for each sampling unit. Sometimes data are not available for each unit but are already summarized by counting the frequency of occurrences of each value, called frequency form data.

Datasets

A dataset is a range of contiguous cells on an Excel worksheet containing data to analyze.

When arranging data on an Excel worksheet you must follow a few simple rules so that Analyse-it works with your data:

1. Title to clearly describe the data. If you do not specify a title, the cell range of the dataset (such as A3:C13) is used to refer to the dataset.

2. A header row containing variable labels. Each variable name should be unique. Units of measurement can be included in the label by enclosing them in brackets after the name.

3. Rows containing the data for each case. The number of rows is only limited by Excel (currently over one million).

4. Columns containing the data for each variable.

5. Optional: Labels in the first column to provide a meaningful name/identifier for each case.

When you use an Analyse-it command, the extent of a dataset is determined by scanning outwards from the active cell to include all surrounding contiguous cells. The extent is known when a blank row or column surrounding the dataset, or the edge of the worksheet, is reached.

Most Excel commands (for example, Sort, Pivot Table) use the same technique to determine the range of cells to operate on. It avoids the need to select often-large ranges of cells using the mouse, which is laborious and error-prone. It also ensures that if you add, or remove, cases or variables, that subsequent analyses automatically reflect any changes to the dataset.

When you analyze data, any data in hidden rows on the worksheet are excluded. This feature lets you easily limit analysis to a subset of the cases in the dataset. You can hide rows manually or use a filter to hide them based on criteria.

You can locate a dataset anywhere on a worksheet, and you can keep multiple datasets on a single worksheet provided you separate them from each other by at least one blank row and column. However, we recommend that you use a separate worksheet for each dataset. Using separate worksheets allows you to name datasets using the Excel worksheet tabs, navigate between datasets using the worksheet tabs, and ensures that filtering a dataset does not affect other datasets on the same worksheet.

Related concepts

Case and frequency form data

Variables

Related tasks

Labeling cases

Analyzing a subset of the data

Variables

A variable is an attribute of an object. The value of a variable can vary from one thing to another.

There are two different types of variables:

- Quantitative (or numeric) - data are numeric values of a quantity.

- Qualitative (or categorical) - data differs only in kind.

Record quantitative data as the numeric value.

Record qualitative data as a numeric coding (for example, 0, 1, 2) or labels (for example, Low, Medium, High). Numeric codings are often preferred as they succinctly represent the values and allow quicker data entry. However, codings can be difficult to understand by anyone other than the person who assigned them. To make their meaning clear, you can also assign a label to each numeric coding.

Data are ordered based on numeric value, numeric coding, or if a numeric coding is not used into alphabetical label order. For some statistical tests, the sort order determines how observations are rank-ordered and can affect the results. Often alphabetic sort order does represent the correct ordering. For example Low, Medium, High is sorted alphabetically into High, Low, Medium which does not represent their correct, natural order. In such cases, you must explicitly set the order before analyzing the data.

Related concepts

Datasets

Measurement scales

Missing values

Related tasks

Ordering categorical data

Assigning labels to categories

Transforming variables

Measurement scales

Five different scales are used to classify measurements based on how much information each measurement conveys.

The different levels of measurement involve different properties of the numbers or symbols that constitute the measurements and also an associated set of permissible transformations.

Measurement scale

Properties

Permissible transformation

Nominal

Two things are assigned the same symbol if they have the same value of the attribute.
For example, Gender (Male, Female); Religion (coded as 0=None, 1=Christian, 2=Buddhist).

Permissible transformations are any one-to-one or many-to-one transformation, although a many-to-one transformation loses information.

Ordinal

Things are assigned numbers such that the order of the numbers reflects an order relation defined on the attribute. Two things x and y with attribute values a(x) and a(y) are assigned numbers m(x) and m(y) such that if m(x) > m(y), then a(x) > a(y).
For example, Moh's scale for the hardness of minerals; academic performance grades (A, B, C, ...).

Permissible transformations are any monotone increasing transformation, although a transformation that is not strictly increasing loses information.

Interval

Things are assigned numbers such that differences between the numbers reflect differences of the attribute. If m(x) - m(y) > m(u) - m(v), then a(x) - a(y) > a(u) - a(v).
For example, Temperature measured in degrees Fahrenheit or Celsius.

Permissible transformations are any affine transformation t(m) = c * m + d, where c and d are constants; another way of saying this is that the origin and unit of measurement are arbitrary.

Ratio

Things are assigned numbers such that differences and ratios between the numbers reflect differences and ratios of the attribute.
For example, Temperature measured in degrees Kelvin scale; Length in centimeters.

Permissible transformations are any linear (similarity) transformation t(m) = c * m, where c is a constant; another way of saying this is that the unit of measurement is arbitrary.

Absolute

Things are assigned numbers such that all properties of the numbers reflect analogous properties of the attribute.
For example, Number of children in a family, Frequency of occurrence.

The only permissible transformation is the identity transformation.

While the measurement scale cannot determine a single best statistical method appropriate for data analysis, it does define which statistical methods are inappropriate. Where possible the use of a variable is restricted when its measurement scale is not appropriate for the analysis. For example, a nominal variable cannot be used in a t-test.

When the measurement scale of a variable is unknown, the scale is inferred from its role in the analysis and the type of data in the variable. If the measurement scale cannot be inferred, you must set the measurement scale.

Related concepts

Variables

Related tasks

Setting the measurement scale of a variable

Missing values

Missing values occur when no data is recorded for an observation; you intended to make an observation but did not. Missing data are a common occurrence and can have a significant effect on the statistical analysis.

Missing values arise due to many reasons. For example, a subject dropped out of the study; a machine fault occurred that prevented a measurement been taken; a subject did not answer a question in a survey; or a researcher made a mistake recording an observation.

A missing value is indicated by an empty cell or a . (full stop) or * (asterisk) in the cell. These values always indicate a missing value, regardless of measurement scale of the variable. Similarly, cell values that are not valid for the measurement scale of the variable are treated as missing. For example, a cell containing text or a #N/A error value is treated as missing for a quantitative measurement scale variable, since neither are valid numeric values.

Some applications, especially older statistical software, use special values such as 99999 to indicate missing values. If you import such data, you should use the Excel Find & Replace command to replace the values with a recognized missing indicator.

In the presence of missing values you often still want to make statistical inferences. Some analyses support techniques such as deletion, imputation, and interpolation to allow the analysis to cope with missing values. However, it is still important before performing an analysis that you understand why the data is missing. When the reason for the missing data is not completely random, the study may be biased, and the statistics may be misleading.

Related concepts

Variables

Case and frequency form data

Most data are available in case form with data for each sampling unit. Sometimes data are not available for each unit but are already summarized by counting the frequency of occurrences of each value, called frequency form data.

Whenever possible you should record data in case form. Case form data can be reduced to frequency form, but frequency form data cannot be reconstructed into case form. You may want to use frequency form data when working with a large amount of data from a database, where you can save computing resources by letting the database server tabulate the data.

Case form dataset layout

Each column is a variable (Height, Eye color), each row is a separate case (Subject) with the values of the variables on that case.

Subject (optional)

Height

Eye color

1

175

Blue

2

180

Blue

3

160

Hazel

4

190

Green

5

180

Green

6

150

Brown

7

140

Blue

8

160

Brown

…

…

…

Frequency form dataset layout

Each column is a variable (Eye color) and a separate column for the number of cases (Frequency), each row is a combination of categories with the frequency count.

Eye color

Frequency

Brown

221

Blue

215

Hazel

93

Green

64

Note: Examples of dataset layouts are included in the Statistical Reference Guide.

Related concepts

Datasets

Setting the measurement scale of a variable

Set the measurement scale of a variable so data analysis can use the variable correctly.

Note: If you have the analysis task pane open, you can quickly set the measurement scale of a variable. In the analysis task pane, click the drop-down menu button alongside the variable, and then click the measurement scale.

1. Activate the dataset worksheet.

2. On the Analyse-it ribbon tab, in the Dataset group, click Dataset.
The dataset task pane opens.

3. In the Variables list, select the variables:

- To select a single variable, click the variable.

- To select multiple variables, click the first variable then hold down the CTRL key and click each additional variable.

- To select a range of variables, click the first variable then hold down the SHIFT key and click the last variable in the range.

The cells of the selected variables highlight on the worksheet. You can also select cells on the worksheet to select the corresponding variables in the list.

4. Click Nominal, Ordinal, Interval, Ratio, or Absolute.

5. Click Apply.
The changes are saved and will be used in future analyses.

Related concepts

Measurement scales

Setting the minimum, maximum and units of a variable

Set the minimum/maximum and units so that the axes of charts are scaled to your preferences.

Note: If you do not set a scale then the chart axis scale will be determined automatically by the data values. If you do specify the scale, but the data values are outside the minimum/maximum then axes are scaled to the next unit. If the unit is too small to show with a label on the axis it is shown as a minor tick mark.

Note: If you have the analysis task pane open you can quickly change the scaling for a variable: right-click over the variable, or click the drop-down menu button alongside the variable selector, and then click Scale.

1. Activate the dataset worksheet.

2. On the Analyse-it ribbon tab, in the Dataset group, click Dataset.
The dataset task pane opens.

3. In the Variable list, select the variables:

- To select a single variable, click the variable.

- To select multiple variables, click the first variable then hold down the CTRL key and click each additional variable.

- To select a range of variables, click the first variable then hold down the SHIFT key and click the last variable in the range.

The cells of the selected variables highlight on the worksheet. You can also select cells on the worksheet to select the corresponding variables in the list.

4. If the measurement scale is not already set, click Interval, Ratio, or Absolute.

5. In the MinimumMaximum and Units edit boxes, optionally, type the value for each. You can provide a minimum value only, a maximum value only, or a minimum, maximum and units. If you provide units then the range (the maximum - minimum) must be a factor of the units (the range must be divisible by the units).

6. Click OK.
Any future or updated analysis will now use the preferred scale where possible.

Ordering categorical data

Set the order of categories for data analysis and output in the analysis report.

1. Activate the dataset worksheet.

2. On the Analyse-it ribbon tab, in the Dataset group, click Dataset.
The dataset task pane opens.

3. In the Variables list, select the variable.

4. Click Nominal or Ordinal to set the measurement scale.
The Categories grid shows the numeric coding and labels used in the data.

5. Clear the Sort Categories check box to allow you to reorder the categories manually.

6. Arrange the categories into order:

- Using drag and drop, click and hold the mouse button down over a category until the drag insertion cursor appears, drag it to reposition it in the list and then release the mouse button.

- Using the arrows, click a category to select it and then use the up and down arrow buttons to reposition it in the list.

7. Click Apply.
The changes are saved and will be used in future analyses.

Setting the number format of a variable

Set the precision of the data so that the results of the analysis show to a suitable number of decimal places.

Note: If you do not set the number format (or you use the Excel General number format), the number of decimal places is determined by the data value with the most decimal places.

1. Activate the dataset worksheet.

2. On the Analyse-it ribbon tab, in the Dataset group, click Dataset.
The dataset task pane opens.

3. In the Variable list, select the variables:

- To select a single variable, click the variable.

- To select multiple variables, click the first variable then hold down the CTRL key and click each additional variable.

- To select a range of variables, click the first variable then hold down the SHIFT key and click the last variable in the range.

The cells of the selected variables highlight on the worksheet. You can also select cells on the worksheet to select the corresponding variables in the list.

4. Click Number format.

5. In the Format Cells window, click Number, set the number of Decimal places, and then click OK.

6. Click OK.
The data is re-formatted.

Related concepts

Presentation of numerical data

Labeling cases

Set a variable to use for labeling data points in plots for easy identification.

- Ensure that the variable containing the labels is the first (leftmost) column in the dataset.

1. Activate the dataset worksheet.

2. On the Analyse-it ribbon tab, in the Dataset group, click Dataset.
The dataset task pane opens.

3. Select the Labels in first column check box.

4. Click Apply.
The changes are saved and will be used in future analyses.

Assigning labels to categories

Assign labels to a numerical coding to make an analysis report more readable.

1. Activate the dataset worksheet.

2. On the Analyse-it ribbon tab, in the Dataset group, click Dataset.
The dataset task pane opens.

3. In the Variables list, select the variable.

4. If the measurement scale is not set, click Nominal or Ordinal.
The Categories grid shows the numeric coding and labels used in the data.

5. In the Name column alongside each coding, type the label.

Note: Names must be unique.

6. Click Apply.
The changes are saved and will be used in future analyses.

Assigning colors/symbols to categories

Assign colors or symbols to highlight observations on plots.

1. Activate the dataset worksheet.

2. On the Analyse-it ribbon tab, in the Dataset group, click Dataset.
The dataset task pane opens.

3. In the Variables list, select the variable.

4. If the measurement scale is not set, click Nominal or Ordinal.
The Categories grid shows the codings used in the data.

5. Select the Assign colors/symbols check box.
The Categories grid shows the Symbol and Color columns.

6. Optional: Click a cell in the Color column and select a preset color.

7. Optional: Click a cell in the Symbol column and select a symbol.

8. Repeat steps 6 through 7 for each coding.

Note: Any categories without an assigned symbol use a square, and any without an assigned color use black. You can assign symbols and colors just to the categories you want to highlight. The others use the default symbol and color.

9. Click Apply.
The changes are saved and will be used in future analyses.

Transforming variables

Transform a variable to normalize, shift, scale or otherwise change the shape of the distribution so that it meets the assumptions of a statistical test.

1. Activate the dataset worksheet.

2. In an empty column adjoining the dataset, enter the transformation function.
For example =LOG(A1)

3. Double-click on the drag handle at the bottom right of the active cell to copy the formula down the entire column.

Related reference

Transform functions

Transform functions

Useful Microsoft Excel functions for transforming data.

Transform

Excel function

Natural log

=LN(cell)

Inverse natural log

=EXP(cell)

Reciprocal

=1 / cell

Square root

=SQRT(cell)

Power

=cell^power

Log base 10

=LOG(cell) or =LOG10(cell)

Inverse log base 10

=10^cell

Log base 2

=LOG(cell, 2)

Inverse log base 2

=2^cell

Related tasks

Transforming variables

Analyzing a subset of the data

Use an Excel filter to show only the rows that match a value or meet a condition. Rows that do not match the condition are hidden temporarily, and subsequent data analysis excludes them. This feature is extremely useful when you want to see how the analysis changes for a different subset of the data.

Note: Prior to version 5.10 the active filter is not saved with the analysis. When you recalculate an analysis the rows currently visible are analyzed, regardless of the filter you used when creating the analysis. From version 5.10 onwards, by default the filter criteria are saved with the analysis and re-applied when you recalculate or make other changes to the analysis. The drop-down arrow next to the Filter command on the ribbon of the active analysis lets you change between Use active filter which always uses the currently active filter when recalculating (pre version 5.10 behavior) and Save Filter & Re-apply (version 5.10 or later behavior) which saves the filter when the analysis is calculated and re-applies it on subsequent recalculation.

1. On the Analyse-it ribbon tab, click Filter to show filter icons on the active dataset.
If you are on an analysis report, the saved filter (if any) will be applied to the dataset. You can modify the filter and return to the analysis and click Recalculate to apply the new filter, or you can start a new analysis to reuse the filter criteria.

2. Click the drop-down button alongside a variable, and then either select the values to match, or apply other criteria.
The save filter feature in version 5.10 or later works with almost all filter conditions: filter by value, by condition, by color/fill/icon, above/below average, top/bottom items, and so on. There is one limitation: filtering to multiple dates. If you filter to more than 2 dates, add-ins such as Analyse-it cannot retrieve the list of values to filter to. This is a limitation of Microsoft Excel and unfortunately it hasn’t been remedied in the last few years so it is low on Microsoft’s priorities. If this is seriously affecting you, please get in touch with Microsoft and ask them to resolve this limitation for add-in developers.

3. Create or recalculate an analysis to see the effect of the filter.

Examples

To filter to Female cases only, click the drop-down button alongside Sex, and then select Female in the list of values to match against:

To filter to cases where the subject is older than 40, click the drop-down button alongside Age, and then click Number Filters > Greater Than, and then enter 40 as the cut-off point:

Chapter 7. Working with analyses

Statistical analyses describe the observed data, make inferences about a population based on a sample of data, fit models that describe the relationship between variables, reduce the dimensionality of large datasets, and much more.

- Analysis reports
An analysis report is a standard Excel worksheet containing statistics and plots calculated by a statistical analysis. You can print, send, move, copy, and save analysis reports like any other worksheet.

- Numerical accuracy
All calculations are performed using double-precision IEEE 754 standard, highly reliable numerical methods, and then tested rigorously.

- Presentation of numerical data
Although all results are computed to high precision, they are presented only to a practical number of decimal places based on the numerical precision of the data.

Creating an analysis

Create a statistical analysis and present the results.

The basic sequence of steps to create an analysis are:

1. Click a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click the analysis command, and then click a specific task command.
The analysis task pane opens.

3. Select the variables to analyze.

4. Add any additional plots, tests or statistics.

5. Set any options for analysis.

6. Click Calculate.
The analysis report opens.

Related concepts

Analysis reports

Analysis reports

An analysis report is a standard Excel worksheet containing statistics and plots calculated by a statistical analysis. You can print, send, move, copy, and save analysis reports like any other worksheet.

Along with the results of the analysis, the report includes a header that describes the analysis, the dataset and variables analyzed, who performed the analysis and when, and the version of Analyse-it used.

Analysis reports are static. If the underlying data changes, the statistics and plots do not update automatically. You must recalculate an analysis to update it. For example, if you exclude or change the value of an observation, add or remove a case, or change the active filter, you must recalculate the analysis to see the effect of the changes.

You can also edit an analysis to make changes such as adding or removing a plot, performing a statistical test, or estimating a parameter.

Note: The analysis report stores a link to the dataset; the link is used to fetch the latest data when you recalculate the analysis. The link to the dataset can become broken if you choose to keep the dataset and analyses in separate workbooks, then move, rename or delete the dataset workbook on disk. When the link is broken, the analysis cannot be changed or updated. To avoid this potential problem we recommend you keep the dataset and associated analyses in the same workbook.

Related tasks

Creating an analysis

Editing an analysis

Creating a new analysis based on an existing analysis

Recalculating an analysis

Printing an analysis report

Editing an analysis

Change an analysis and update the report.

1. Activate the report worksheet.

2. On the Analyse-it ribbon tab, in the Report group, click Edit.
The analysis task pane opens.

3. Change the variables, set the analysis options, or add/remove tasks.

4. Click Recalculate.
The analysis report updates.

Recalculating an analysis

Recalculate an analysis when the data has changed, and you want to update the report to reflect those changes.

Note: Be aware that when you recalculate an analysis, the current analysis report worksheet is deleted and replaced with a new worksheet. If you have made changes to the layout or formatting of the analysis report, the changes are lost.

1. Activate the report worksheet.

2. On the Analyse-it ribbon tab, in the Report group, click Recalculate.

Note: Prior to version 5.10 the active filter is not saved with the analysis. When you recalculate an analysis the rows currently visible are analyzed, regardless of the filter you used when creating the analysis. From version 5.10 onwards, by default the filter criteria are saved with the analysis and re-applied when you recalculate or make other changes to the analysis. The drop-down arrow next to the Filter command on the ribbon of the active analysis lets you change between Use active filter which always uses the currently active filter when recalculating (pre version 5.10 behavior) and Save Filter & Re-apply (version 5.10 or later behavior) which saves the filter when the analysis is calculated and re-applies it on subsequent recalculation.

The analysis recalculates and the report updates.

Excluding observations from analysis

Exclude observations, such as outliers or influential observations, from analysis to see their effect on the results.

Before you exclude observations from the analysis, you should fully investigate them. Rather than a nuisance, outliers can sometimes be the most interesting and insightful observations in the data.

1. Activate the dataset worksheet.

2. Select the cells containing the observations to exclude/include.

3. On the Analyse-it ribbon tab, in the Dataset group, click Include / Exclude.
The observations will be excluded (or re-included) in future analyses.

Observations to exclude from the analysis are enclosed in square brackets (for example, [65] or [Male]) in the dataset. Excluded observations are treated the same as missing values by an analysis.

We recommend you attach an Excel comment to the cell to document the reason for excluding the observation.

Related tasks

Recalculating an analysis

Creating a new analysis based on an existing analysis

Change an analysis and output the results to a new report.

1. Activate the report worksheet.

2. On the Analyse-it ribbon tab, in the Report group, click Clone.
The analysis task pane opens with the existing analysis options.

3. Change the variables, set the analysis options, or add/remove tasks.

4. Click Calculate.
A new analysis report opens. The original analysis report is left unchanged.

Printing an analysis report

Print a publication-quality report of the analysis.

Note: Reports include page breaks, so tables and plots are not split across multiple pages when printed. The paper size for the default Windows printer determines where to place page breaks. If you are in the US or Canada, the paper size for your printer will most likely be Letter (8.5” x 11”). If the paper size for your printer is not Letter, then the A4 paper size (210mm x 297mm) is used.

1. Activate the report worksheet.

2. On the Analyse-it ribbon tab, in the Report group, click Print.
The Print Preview window opens.

3. Choose the printer options.

4. Click Print.

Related tasks

Printing in color / black & white

Printing a single chart on a full page

Printing in color / black & white

Set whether to print reports in color or black and white.

Note: By default, reports print in black and white, even if you have a color printer.

1. Activate the report worksheet.

2. On the Analyse-it ribbon tab, in the Report group, click the Print drop-down button, and then click Color or Black and White.
Your choice is saved as the default. In future, you can just click Print to print using the new default.

3. Choose the printer options.

4. Click Print.

Printing a single chart on a full page

Print a single high-resolution chart scaled to fill a single page.

1. Activate the report worksheet.

2. Click on the plot to select it.
The plot shows a thick border around it to indicate that it is selected.

3. On the Analyse-it ribbon tab, in the Report group, click Print.
The Print Preview window opens.

4. Choose the printer options.

5. Click Print.

Related tasks

Printing in color / black & white

Showing the dataset analyzed

View the data analyzed and exclude observations, filter, or make other changes to it.

1. Activate the report worksheet.

2. On the Analyse-it ribbon tab, in the Report group, click Go to Dataset.
The workbook containing the dataset opens.

Changing the user name on analysis reports

Set the Excel user name to show on analysis reports.

1. Open the Options window:

- In Excel 2007, click the Office button, and then click Excel Options.

- In Excel 2010, click File, and then click Options.

2. Change the User name.

3. Click OK.
The changes are saved and will be used in future analyses.

Numerical accuracy

All calculations are performed using double-precision IEEE 754 standard, highly reliable numerical methods, and then tested rigorously.

To ensure that Analyse-it produces accurate results it does not use any Excel mathematical, statistical or distribution functions. Many of these functions in early versions of Microsoft Excel used poor algorithms that often produced inaccurate results. Functions like STDEV and LINEST were affected, as were add-ins that used those functions. Microsoft has now addressed most of the problems, though you may still hear warnings against using Microsoft Excel for statistical analysis.

To avoid any of these problems Analyse-it uses reliable numerical algorithms and all calculations are performed using double-precision (IEEE 754 standard, effectively 15 significant digits). We then verify the statistics are correct, and remain so, throughout the development and release process using an in-house library of thousands of validation unit-tests.

Related concepts

Presentation of numerical data

Presentation of numerical data

Although all results are computed to high precision, they are presented only to a practical number of decimal places based on the numerical precision of the data.

To decide how many decimal places to present the results to Analyse-it uses the Excel number format you apply to the data. From the format, the precision you are working with is inferred, and therefore the precision you likely expect in the presentation of the statistics. Statistics such as the mean, median, and standard deviation are presented to one more decimal place than the source data.

Although the statistics are presented to only a few decimal places, the Excel cell contains the full precision value. You should use the full precision value to minimize error when using it in another calculation.

If you are preparing an analysis for presentation you can set the Excel number format applied to a cell containing a statistic on the analysis report. Do not format results if you intend to edit or recalculate the analysis as your formatting changes will be lost when you update the report.

Related concepts

Numerical accuracy

Related tasks

Showing a statistic to full precision

Setting the number format of a variable

Showing a statistic to full precision

Use the full precision of a statistic when you want to use it in further calculations.

1. If the Excel formula bar is not visible, on the View ribbon tab, in the Show group, select the Formula bar check box.

2. Click in the cell containing the statistic.
The Excel formula bar shows the value with up to 15 significant digits.

Related concepts

Presentation of numerical data

Chapter 8. Feedback and crash reports

Suggestions, bug, and crash reports you send to our development team help shape the future of Analyse-it.

The feedback you send helps us to identify, diagnose and fix application crashes and prioritize development time toward the most requested features. When sending feedback try to include as much information as possible to help us understand the issue. We read all your feedback, but we only respond to you if we need more information about your report.

Do not use feedback for support requests. For those, contact us directly support@analyse-it.com

Note: Feedback and crash reporting use your internet connection. If your computer is behind a firewall, ensure the TCP port 443 (HTTPS) on the firewall is open for communication to secure.analyse-it.com.

- Sending feedback
Submit your ideas for improvements, feature requests, and bug reports.

- Submitting crash reports
Submit a crash report whenever you experience a problem so that we can try to solve it.

Sending feedback

Submit your ideas for improvements, feature requests, and bug reports.

1. On the Analyse-it ribbon tab, click Analyse-it, and then click Send Feedback.
The Send Feedback window opens.

2. Enter an E-mail address where we can contact you.
We respect your privacy at all times. We do not collect personal information, and we only use your e-mail address if we need to contact you about your problem or suggestion.

3. Enter a Subject and Feedback to describe your comment, suggestion, or bug report. Include as much information as possible in the feedback, including the steps to reproduce the problem if applicable.

4. Select Include a snapshot of my desktop if it helps to show the problem or suggestion you are making.

5. Select Include the file I am working with to include the active Excel workbook if it contains data that helps to show the problem or suggestion you are making.
We treat any data received as private and confidential. It is used only for the purpose you submit it, and we delete it when no longer needed.

6. Click Send.
Your feedback is sent using your internet connection.

Related concepts

Feedback and crash reports

Submitting crash reports

Submit a crash report whenever you experience a problem so that we can try to solve it.

If Analyse-it crashes or experiences a problem, it creates a crash report and prompts you to send it to us for investigation. Crash reports include technical information (application state, hardware configuration, and stack traces) to help us diagnose and fix the problem. Before Analyse-it submits the crash report it checks if a fix is available. If so, Analyse-it gives you details on how to fix the problem. If no fix is available, you should submit the crash report to help us improve the application. Even if you experience a crash that seems similar to previous crashes you should still submit it as each report often provides new insight or perspective on a crash.

1. If Analyse-it detects a serious problem, the Crash Report window opens.

2. Enter an E-mail address where we can contact you.
We respect your privacy at all times. We do not collect any personal information, and we only use your e-mail address if we need to contact you about the crash report.

3. Enter Feedback to describe what you were doing when the crash occurred and, if possible, the steps to reproduce the problem.

4. Select Include a snapshot of my desktop if it helps to show the problem.

5. Select Include the file I am working with to include the active Excel workbook if it contains data that helps to show the problem.
We treat any data received as private and confidential. It is used only for the purpose you submit it, and we delete it when no longer needed.

6. Click Send.
Your crash report is sent using your internet connection.

After submitting the crash report, Analyse-it attempts to recover so you can save your work and continue using the software. If the crash was serious and causes repeated subsequent crashes, you should restart Microsoft Excel and Analyse-it.

Related concepts

Feedback and crash reports

Part III. Statistical Reference Guide

Chapter 9. Distribution

Distribution analysis describes the distribution of a variable and makes inferences about population parameters.

- Continuous distributions
A continuous distribution describes a variable that can take on any numeric value (for example, weight, age, or height).

- Discrete distributions
A discrete distribution describes a variable that can only take discrete values (for example, the number of male and females, or the number of people with a specific eye color).

Continuous distributions

A continuous distribution describes a variable that can take on any numeric value (for example, weight, age, or height).

- Univariate descriptive statistics
Descriptive statistics provide information about the central location (central tendency), dispersion (variability or spread), and shape of the distribution.

- Univariate plot
A univariate plot shows the data and summarizes its distribution.

- Frequency distribution
A frequency distribution reduces a large amount of data into a more easily understandable form.

- Normality
Normality is the assumption that the underlying random variable is normally distributed, or approximately so.

- Inferences about distribution parameters
Inferences about the parameters of a distribution are made using a random sample of data drawn from the population of interest.

Univariate descriptive statistics

Descriptive statistics provide information about the central location (central tendency), dispersion (variability or spread), and shape of the distribution.

There are many measures of central location, dispersion, and shape:

Statistic

Purpose

N

The number of non-missing values in a set of data.

Sum

The sum of the values in a set of data.

Mean

Measure the central tendency using the arithmetic mean.

Harmonic mean

Measure the central tendency using the reciprocal of the arithmetic mean of the reciprocals.
Useful when the values are rates and ratios.

Geometric mean

Measure the central tendency using the product of the values.
Useful when the values are percentages.

Variance

Measure the amount of variation using the squared deviation of a variable from its mean.

Standard deviation

Measure the amount of variation using the square root of the variance.

CV% / RSD

Measure the spread relative to its expected value (standard deviation divided by the mean).
Also known as the coefficient of variation or relative standard deviation.

Skewness

Measure the "sideness" or symmetry of the distribution.
Skewness can be positive or negative. Negative skew indicates that the tail on the left side of the distribution is longer or fatter than the right side. Positive skew indicates the converse, the tail on the right side is longer or fatter than the left side. A value of zero indicates the tails on both sides balance; this is the case for symmetric distributions, but also for asymmetric distributions where a short fat tail balances out a long thin tail.
Uses the Fisher-Pearson standardized moment coefficient G_1 definition of sample skewness.

Kurtosis

Measure the "tailedness" of the distribution. That is, whether the tails are heavy or light. Excess kurtosis is the kurtosis minus 3 and provides a comparison to the normal distribution.
Positive excess kurtosis (called leptokurtic) indicates the distribution has fatter tails than a normal distribution. Negative excess kurtosis (called platykurtic) indicates the distribution has thinner tails than a normal distribution.
Uses the G_2 definition of the sample excess kurtosis

Median

Measure the central tendency using the middle value in a set of data.

Minimum

Smallest value in a set of data.

Maximum

Largest value in a set of data.

Range

The difference between the maximum and minimum.

1st quartile

The middle value between the smallest value and median in a set of data.

3rd quartile

The middle value between the median and largest value in a set of data.

Interquartile range

Measure the spread between the 1st and 3rd quartile.

Mode

The value that appears the most in a set of data.

Quantiles

A set of values that divide the range of the distribution into contiguous intervals defined by probabilities.

For normally distributed data, the mean and standard deviation provide the best measures of central location and dispersion.

For data with a non-normal or highly-skewed distribution, or data with extreme values, the median and the first and third quartiles provide better measures of central location and dispersion. When the distribution of the data is symmetric, the inter-quartile range (IQR) is a useful measure of dispersion. Quantiles further describe the distribution of the data, providing an interval containing a specified proportion (for example, 95%) of the data or by breaking the data into intervals each containing a proportion of the data (for example, deciles each containing 10% of the data).

Related tasks

Calculating univariate descriptive statistics

Calculating univariate descriptive statistics, by group

Calculating univariate descriptive statistics

Use univariate descriptive statistics to describe a quantitative variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution, and then click the statistics to show:

Option

Description

Mean and Moments

Show mean, variance, standard deviation, skewness, and kurtosis.

Median and Quartiles

Show minimum, maximum, median, and quartiles.

Quantiles

Show quantiles.

The analysis task pane opens.

3. In the Y drop-down list, select the variable.

4. Optional: To customize the statistics to show, click Customize... and then select or clear the appropriate check boxes. To save the options as the defaults to use for future analyses, click Save as Defaults.

5. Click Calculate.

Related concepts

Univariate descriptive statistics

Related reference

Dataset layout

Univariate plot

A univariate plot shows the data and summarizes its distribution.

- Dot plot
A dot plot, also known as a strip plot, shows the individual observations.

- Box plot
A box plot shows the five-number summary of the data – the minimum, first quartile, median, third quartile, and maximum. An outlier box plot is a variation of the skeletal box plot that also identifies possible outliers.

- Mean plot
A mean plot shows the mean and standard deviation of the data.

Dot plot

A dot plot, also known as a strip plot, shows the individual observations.

A dot plot gives an indication of the spread of the data and can highlight clustering or extreme values.

Related concepts

Box plot

Mean plot

Related tasks

Creating a univariate plot

Box plot

A box plot shows the five-number summary of the data – the minimum, first quartile, median, third quartile, and maximum. An outlier box plot is a variation of the skeletal box plot that also identifies possible outliers.

Skeletal box plot

A skeletal box plot shows the median as a line, a box from the 1st to 3rd quartiles, and whiskers with end caps extending to the minimum and maximum. Optional notches in the box represent the confidence interval around the median.

Outlier box plot

An outlier box plot is a variation of the skeletal box plot, but instead of extending to the minimum and maximum, the whiskers extend to the furthest observation within 1.5 x IQR from the quartiles. Possible near outliers (orange plus symbol) are identified as observations further than 1.5 x IQR from the quartiles, and possible far outliers (red asterisk symbol) as observations further than 3.0 x IQR from the quartiles. You should investigate each possible outlier before deciding whether to exclude it, as even in normally distributed data, an outlier box plot identifies approximately 0.7% of observations as possible outliers.

Quantile box plot

A quantile box plot is a variation on the skeletal box plot and shows the whiskers extending to specific quantiles rather than the minimum and maximum value.

Related concepts

Dot plot

Mean plot

Related tasks

Creating a univariate plot

Mean plot

A mean plot shows the mean and standard deviation of the data.

A line or dot represents the mean. A standard error or confidence interval measures uncertainty in the mean and is represented as either an error bar or diamond.

An optional error bar or band represents the standard deviation. The standard deviation gives the impression that the data is from a normal distribution centered at the mean value, with most of the data within two standard deviations of the mean. Therefore, the data should be approximately normally distributed. If the distribution is skewed, the plot is likely to mislead.

Related concepts

Box plot

Dot plot

Related tasks

Creating a univariate plot

Creating a univariate plot

Plot a dot plot, box plot, or mean plot to visualize the distribution of a single quantitative variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution > Univariate, and then click the plot type.
The analysis task pane opens.

3. In the Y drop-down list, select the variable.

4. Optional: To overlay more than one plot type on the same chart, on the Analyse-it ribbon tab, in the Distribution group, click Univariate, and then click an additional plot type.

5. Optional: In the analysis task pane, customize the options for the plot.

6. Click Calculate.

Related concepts

Box plot

Dot plot

Mean plot

Related reference

Dataset layout

Frequency distribution

A frequency distribution reduces a large amount of data into a more easily understandable form.

- Cumulative distribution function plot
A cumulative distribution function (CDF) plot shows the empirical cumulative distribution function of the data.

- Histogram
A histogram shows the distribution of the data to assess the central tendency, variability, and shape.

Cumulative distribution function plot

A cumulative distribution function (CDF) plot shows the empirical cumulative distribution function of the data.

The empirical CDF is the proportion of values less than or equal to X. It is an increasing step function that has a vertical jump of 1/N at each value of X equal to an observed value. CDF plots are useful for comparing the distribution of different sets of data.

Related tasks

Creating a CDF plot

Creating a CDF plot

Plot a CDF to visualize the shape of the distribution of a quantitative variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution > CDF, and then click the plot type.
The analysis task pane opens.

3. In the Y drop-down list, select the variable.

4. Optional: To show the confidence band, in the Confidence band edit box, type the confidence level.

5. Click Calculate.

Related concepts

Cumulative distribution function plot

Related reference

Dataset layout

Histogram

A histogram shows the distribution of the data to assess the central tendency, variability, and shape.

A histogram for a quantitative variable divides the range of the values into discrete classes, and then counts the number of observations falling into each class interval. The area of each bar in the histogram is proportional to the frequency in the class. When the class widths are equal, the height of the bar is also proportional to the frequency in the class.

Choosing the number of classes to use can be difficult as there is no "best," and different class widths can reveal or hide features of the data. Scott's and Freedman-Diaconis' rules provide a default starting point, though sometimes particular class intervals make sense for a particular problem.

The histogram reveals if the distribution of the data is normal, skewed (shifted to the left or right), bimodal (has more than one peak) and so on. Skewed data can sometimes be transformed to normal using a transformation. Bi-modality often indicates that there is more than one underlying population in the data. Individual bars distanced from the bulk of the data can indicate the presence of an outlier.

Related concepts

Frequency table

Related tasks

Creating a histogram

Creating a histogram

Plot a histogram to visualize the distribution of a quantitative variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution > Histogram, and then click the plot type.
The analysis task pane opens.

3. In the Y drop-down list, select the variable.

4. In the Plot drop-down list, select the frequency scale.

5. Optional: To label the bars with the frequencies, select the Label bars check box.

6. Optional: To show the frequency table, select the Frequency table check box.

7. Optional: To customize the classes of the frequency distribution, in the Start at, Width, and Classes edit boxes, type the values to form the classes.

8. Click Calculate.

Related concepts

Histogram

Related reference

Dataset layout

Normality

Normality is the assumption that the underlying random variable is normally distributed, or approximately so.

In some cases, the normality of the data itself may be important in describing a process that generated the data. However, in many cases, it is hypothesis tests and parameter estimators that rely on the assumption of normality, although many are robust against moderate departures in normality due to the central limit theorem.

- Normal distribution
A normal (or Gaussian) distribution is a continuous probability distribution that has a bell-shaped probability density function. It is the most prominent probability distribution in used statistics.

- Normal probability (Q-Q) plot
A normal probability plot, or more specifically a quantile-quantile (Q-Q) plot, shows the distribution of the data against the expected normal distribution.

- Normality hypothesis test
A hypothesis test formally tests if the population the sample represents is normally-distributed.

- Central limit theorem and the normality assumption
Due to central limit theory, the assumption of normality implied in many statistical tests and estimators is not a problem.

Normal distribution

A normal (or Gaussian) distribution is a continuous probability distribution that has a bell-shaped probability density function. It is the most prominent probability distribution in used statistics.

Normal distributions are a family of distributions with the same general symmetric bell-shaped curve, with more values concentrated in the middle than in the tails. Two parameters describe a normal distribution, the mean, and standard deviation. The mean is the central location (the peak), and the standard deviation is the dispersion (the spread). Skewness and excess kurtosis are zero for a normal distribution.

The normal distribution is the basis of much statistical theory. Statistical tests and estimators based on the normal distribution are often more powerful than their non-parametric equivalents. When the distribution assumption can be met they are preferred, as the increased power lets you use a smaller sample size to detect the same difference.

Related concepts

Normality hypothesis test

Normal probability (Q-Q) plot

Normal probability (Q-Q) plot

A normal probability plot, or more specifically a quantile-quantile (Q-Q) plot, shows the distribution of the data against the expected normal distribution.

For normally distributed data, observations should lie approximately on a straight line. If the data is non-normal, the points form a curve that deviates markedly from a straight line. Possible outliers are points at the ends of the line, distanced from the bulk of the observations.

Related concepts

Normality hypothesis test

Normal distribution

Related tasks

Creating a normal probability plot

Creating a normal probability plot

Plot a Normal (Q-Q) plot to subjectively assess the normality of a quantitative variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution > Normal, and then click the plot type.
The analysis task pane opens.

3. In the Y drop-down list, select the variable.

4. Optional: To label the points using the case label, if set, otherwise the observed value, select the Label points check box.

5. Optional: To show the normal distribution and confidence band, select the Normal distribution line check box, and, if required, in the Confidence band edit box, type the level of confidence.

6. Click Calculate.

Related concepts

Normal probability (Q-Q) plot

Related reference

Dataset layout

Normality hypothesis test

A hypothesis test formally tests if the population the sample represents is normally-distributed.

The null hypothesis states that the population is normally distributed, against the alternative hypothesis that it is not normally-distributed. If the test p-value is less than the predefined significance level, you can reject the null hypothesis and conclude the data are not from a population with a normal distribution. If the p-value is greater than the predefined significance level, you cannot reject the null hypothesis.

Note that small deviations from normality can produce a statistically significant p-value when the sample size is large, and conversely it can be impossible to detect non-normality with a small sample. You should always examine the normal plot and use your judgment, rather than rely solely on the hypothesis test. Many statistical tests and estimators are robust against moderate departures in normality due to the central limit theorem.

Related concepts

Normal probability (Q-Q) plot

Normal distribution

Hypothesis testing

Central limit theorem and the normality assumption

Related tasks

Testing the normality of a distribution

Related reference

Tests for normality

Tests for normality

Tests for normality of a distribution and the type of non-normality detected.

Test

Purpose

Shapiro-Wilk

Test if the distribution is normal.
A powerful test that detects most departures from normality when the sample size ≤ 5000.

Anderson-Darling

Test if the distribution is normal.
A powerful test that detects most departures from normality.

Kolmogorov-Smirnov

Test if the distribution is normal.
Mainly of historical interest and for educational use only.

Related concepts

Normality hypothesis test

Related tasks

Testing the normality of a distribution

Testing the normality of a distribution

Test if a distribution is normal.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution, and then click the hypothesis test.
The analysis task pane opens.

3. In the Y drop-down list, select the variable.

4. In the Hypotheses drop-down list, select the null and alternative hypothesis.

5. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

6. Click Calculate.

Related concepts

Normality hypothesis test

Related reference

Dataset layout

Tests for normality

Central limit theorem and the normality assumption

Due to central limit theory, the assumption of normality implied in many statistical tests and estimators is not a problem.

The normal distribution is the basis of much statistical theory. Hypothesis tests and interval estimators based on the normal distribution are often more powerful than their non-parametric equivalents. When the distribution assumption can be met they are preferred as the increased power means a smaller sample size can be used to detect the same difference.

However, violation of the assumption is often not a problem, due to the central limit theorem. The central limit theorem states that the sample means of moderately large samples are often well-approximated by a normal distribution even if the data are not normally distributed. For many samples, the test statistic often approaches a normal distribution for non-skewed data when the sample size is as small as 30, and for moderately skewed data when the sample size is larger than 100. The downside in such situations is a reduction in statistical power, and there may be more powerful non-parametric tests.

Sometimes a transformation such as a logarithm can remove the skewness and allow you to use powerful tests based on the normality assumption.

Related concepts

Normality

Inferences about distribution parameters

Inferences about the parameters of a distribution are made using a random sample of data drawn from the population of interest.

- Point and interval estimation
Estimation is the process of making inferences from a sample about an unknown population parameter. An estimator is a statistic that is used to infer the value of an unknown parameter.

- Parameter estimate
A parameter estimate is either a point or interval estimate of an unknown population parameter.

- Hypothesis testing
Hypothesis testing is the formal process of making inferences from a sample whether or not a statement about the population appears to be true.

- Parameter hypothesis test
A hypothesis test formally tests if a population parameter is different to a hypothesized value.

- Parameter equivalence hypothesis test
An equivalence hypothesis test formally tests if a population parameter is equivalent to a hypothesized value, that is, practically the same.

Point and interval estimation

Estimation is the process of making inferences from a sample about an unknown population parameter. An estimator is a statistic that is used to infer the value of an unknown parameter.

A point estimate is the best estimate, in some sense, of the parameter based on a sample. It should be obvious that any point estimate is not absolutely accurate. It is an estimate based on only a single random sample. If repeated random samples were taken from the population, the point estimate would be expected to vary from sample to sample.

A confidence interval is an estimate constructed on the basis that a specified proportion of the confidence intervals include the true parameter in repeated sampling. How frequently the confidence interval contains the parameter is determined by the confidence level. 95% is commonly used and means that in repeated sampling 95% of the confidence intervals include the parameter. 99% is sometimes used when more confidence is needed and means that in repeated sampling 99% of the intervals include the true parameter. It is unusual to use a confidence level of less than 90% as too many intervals would fail to include the parameter. Likewise, confidence levels larger than 99% are not used often because the intervals become wider the higher the confidence level and therefore require large sample sizes to make usable intervals.

Many people misunderstand confidence intervals. A confidence interval does not predict with a given probability that the parameter lies within the interval. The problem arises because the word confidence is misinterpreted as implying probability. In frequentist statistics, probability statements cannot be made about parameters. Parameters are fixed, not random variables, and so a probability statement cannot be made about them. When a confidence interval has been constructed, it either does or does not include the parameter.

In recent years the use of confidence intervals has become more common. A confidence interval provides much more information than just a hypothesis test p-value. It indicates the uncertainty of an estimate of a parameter and allows you to consider the practical importance, rather than just statistical significance.

Confidence intervals and hypothesis tests are closely related. Most introductory textbooks discuss how confidence intervals are equivalent to a hypothesis test. When the 95% confidence interval contains the hypothesized value the hypothesis test is statistically significant at the 5% significance level, and when it does not contain the value the test is not significant. Likewise, with a 99% confidence interval and hypothesis test at the 1% significance level. A confidence interval can be considered as the set of parameter values consistent with the data at some specified level, as assessed by testing each possible value in turn.

Th relationship between hypothesis tests and confidence interval only holds when the estimator and the hypothesis test both use the same underlying evidence function. When a software package uses different evidence functions for the confidence interval and hypothesis test, the results can be inconsistent. The hypothesis test may be statistically significant, but the confidence interval may include the hypothesized value suggesting the result is not significant. Where possible the same underlying evidence function should be used to form the confidence interval and test the hypotheses. Be aware that not many statistical software packages follow this rule!

The following example illustrates the use of a point estimate, confidence interval and hypothesis test in making inferences.
A scientist might study the difference in blood cholesterol between a new drug treatment and a placebo. Improvements in cholesterol greater than 20mg/dL would be considered practically important, and lead to a change in the treatment of patients, but smaller differences would not. The possible outcomes of the study in terms of a point-estimate, confidence interval estimate, and hypothesis test might be:

Point estimate

95% confidence interval

Hypothesis test (5% significance level)

Interpretation

0 mg/dL

-5 to 5 mg/dL

Not significant

The confidence interval is within the range of no practical difference, 20mg/dL.
There is clear evidence the treatment does not produce a difference of practical importance.

10 mg/dL

-5 to 25 mg/dL

Not significant

Part of the confidence interval lies outside the range of no practical difference, 20mg/dL.
Although the hypothesis test is not significant, there may be an important practical difference, though a larger sample size is required to make any sharper inferences.

10 mg/dL

5 to 15 mg/dL

Significant

The confidence interval excludes the null (zero) difference but still lies within the range of no practical difference, 20mg/dL.
Although the hypothesis test is statistically significant, the difference is of no practical importance.

30 mg/dL

25 to 45 mg/dL

Significant

The confidence interval excludes the null difference and lies outside the range of no practical difference, 20mg/dL.
The hypothesis test is statistically significant, and the difference is of practical importance.

Related concepts

Hypothesis testing

Parameter estimate

A parameter estimate is either a point or interval estimate of an unknown population parameter.

A point estimate is a single value that is the best estimate of the true unknown parameter; a confidence interval is a range of values and indicates the uncertainty of the estimate.

Related concepts

Point and interval estimation

Related tasks

Estimating the parameters of a distribution

Related reference

Estimators for the central location parameter of a distribution

Estimators for the dispersion parameter of a distribution

Estimators for the central location parameter of a distribution

Estimators for the central location parameter and their properties and assumptions.

Estimator

Purpose

Mean

Estimate the population mean using the sample mean estimator.

Median

Estimate the middle of the distribution using the sample median estimator.

Hodges-Lehmann pseudo-median

Estimate the population pseudo-median using the Hodges-Lehman pseudo-median estimator.
The pseudo-median is equivalent to the mean/median when the distribution is symmetric.

Related concepts

Parameter estimate

Related tasks

Estimating the parameters of a distribution

Estimators for the dispersion parameter of a distribution

Estimators for the dispersion parameter and their properties and assumptions.

Estimator

Purpose

Standard deviation

Estimate the population standard deviation using the sample standard deviation estimator.
The sample standard deviation is a biased estimator of the population standard deviation.

Variance

Estimate the population variance using the sample variance estimator.
The sample variance is an unbiased estimator of the population variance.

Related concepts

Parameter estimate

Related tasks

Estimating the parameters of a distribution

Estimating the parameters of a distribution

Estimate the mean/median/variance of a distribution.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution, and then click the parameter estimator.
The analysis task pane opens.

3. In the Y drop-down list, select the variable.

4. In the Confidence interval edit box, type the confidence level as a percentage, or type - to suppress the confidence interval, and then in the drop-down list, select the confidence bounds.

5. In the Method drop-down list, select the interval estimator.

6. Click Calculate.

Related concepts

Parameter estimate

Related reference

Dataset layout

Estimators for the central location parameter of a distribution

Estimators for the dispersion parameter of a distribution

Hypothesis testing

Hypothesis testing is the formal process of making inferences from a sample whether or not a statement about the population appears to be true.

A hypothesis test is a method of making decisions. You must state a null hypothesis and an alternative hypothesis to perform a hypothesis test. The null hypothesis states what the study is intending to reject and disprove. The alternative hypothesis is usually the negation of the null and states what the study is trying to prove.

When the hypotheses have been stated a statistical test calculates a test statistic and p-value. The p-value is the probability of obtaining a test statistic at least as extreme as that observed when the null hypothesis is true. It is a measure of evidence against the null hypothesis. When the p-value is small, the data are unlikely to have occurred if the null hypothesis is true so you can reject the null hypothesis and accept the alternative hypothesis. When the p-value is large you cannot reject the null hypothesis; there is insufficient evidence against it. It is not possible to prove the null hypothesis, only disprove it. The p-value does not allow you to make any statements about the probability of the null hypothesis been true, it is a statement based on the observing the data given the null hypothesis is true.

Often a fixed significance level (denoted by the lower case Greek symbol alpha) is used to decide whether the test is statistically significant or not. The significance level is the probability of rejecting the null hypothesis when it is true. When the p-value is less than the significance level, you can declare the test statistically significant. A 5% significance level is typical, which implies there is a 5% chance of wrongly rejecting the null hypothesis when in fact it is true. If more certainty is required, use a 1% significance level. Regardless, you should always report the p-value rather than just a statement of statistically significant, or not.

It is important to remember that a statistically significant test does not imply practically important. The difference might be so small as to be practically useless even though it was statistically significant. Alternatively, the sample size may have been so small that a hypothesis test was not powerful enough to detect anything but a huge difference as statistically significant. It is, therefore, essential that you always interpret the p-value together with a point and interval estimate of the parameter or effect size.

Related concepts

Point and interval estimation

Parameter hypothesis test

A hypothesis test formally tests if a population parameter is different to a hypothesized value.

The null hypothesis states that the parameter is equal to the hypothesized value, against the alternative hypothesis that it is not equal to (or less than, or greater than) the hypothesized value. When the test p-value is small, you can reject the null hypothesis and conclude the sample is not from a population with the parameter equal to the hypothesized value.

Related concepts

Hypothesis testing

Related tasks

Testing distribution parameters

Related reference

Tests for the central location parameter of a distribution

Tests for the dispersion parameter of a distribution

Parameter equivalence hypothesis test

An equivalence hypothesis test formally tests if a population parameter is equivalent to a hypothesized value, that is, practically the same.

An typical parameter hypothesis test can never prove that the parameter is equal to the hypothesized value, it can only ever disprove the null hypothesis of equality. By contrast, an equivalence hypothesis test is of interest when the purpose is to prove that the parameter is equivalent to a hypothesized value, that is the difference is less than some small negligible effect size. An equivalence hypothesis test therefore constructs the null hypothesis of non-equivalence and the goal is to prove the parameter is the equivalent to the hypothesized value.

The null hypothesis states that the parameter is not equivalent to the hypothesized value, against the alternative hypothesis that it is equivalent within the equivalence interval. The hypothesis is tested as a composite of two one-sided t-tests (TOST), H0_1 tests the hypothesis that parameter is less than the lower bound of the equivalence interval, test H0_2 that the parameter is greater than the upper bounds of the equivalence interval. The p-value is the greater of the two one sided t-test p-values. When the test p-value is small, you can reject the null hypothesis and conclude the sample is from a population with the parameter practically equivalent to the hypothesized value.

Related tasks

Testing equivalence of distribution parameters

Related reference

Tests for the central location parameter of a distribution

Tests for the central location parameter of a distribution

Tests for the location parameter of a distribution and their properties and assumptions.

Test

Purpose

Z

Test if the mean is equal to a hypothesized value when the population standard deviation is known.

Student's t

Test if the mean is equal to a hypothesized value.
Assumes the population is normally distributed. Due to the central limit theorem, the test may still be useful when this assumption is not true if the sample size is moderate. However, the Wilcoxon test may be more powerful in this situation.

Wilcoxon

Test if the mean/median is equal to a hypothesized value.
Assumes the population distribution is symmetric. The mean, median, and pseudo-median are equivalent when the distribution is symmetric. Therefore, the hypotheses can be stated in terms of any of these parameters.

Sign

Test if the median is equal to a hypothesized value.
Has few assumptions, but lacks power compared to the Wilcoxon and Student's t test.

TOST (two-one-sided t-test)

Test if the mean is equivalent to a hypothesized value within the equivalence bounds that specify the smallest effect size of interest.
Assumes the population is normally distributed. Due to the central limit theorem, the test may still be useful when this assumption is not true if the sample size is moderate.

Related concepts

Parameter hypothesis test

Central limit theorem and the normality assumption

Parameter equivalence hypothesis test

Related tasks

Testing distribution parameters

Testing equivalence of distribution parameters

Tests for the dispersion parameter of a distribution

Tests for the dispersion parameter of a distribution and their properties and assumptions.

Test

Purpose

X²

Test if the variance/standard deviation is equal to a hypothesized value.
Assumes the population is normally distributed. When this assumption is not true, you should be cautious of using the test, as it is extremely sensitive to deviations from normality.

Related concepts

Parameter hypothesis test

Related tasks

Testing distribution parameters

Testing distribution parameters

Test if the mean/median/variance of a distribution is different to the hypothesized value.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution, and then click the hypothesis test.
The analysis task pane opens.

3. In the Y drop-down list, select the variable.

4. In the Hypotheses drop-down list, select the null and alternative hypothesis.

5. In the Hypothesized value edit box, type the expected value of the parameter under the null hypothesis.

6. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

7. Click Calculate.

Related concepts

Parameter hypothesis test

Related reference

Dataset layout

Tests for the central location parameter of a distribution

Tests for the dispersion parameter of a distribution

Testing equivalence of distribution parameters

Test if the mean of a distribution is equivalent to the hypothesized value.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution, and then click TOST.
The analysis task pane opens.

3. In the Y drop-down list, select the variable.

4. In the Hypothesized value edit box, type the expected value of the parameter under the null hypothesis.

5. In the Hypothesized value edit box, type the expected value of the parameter, and then in the Equivalence bounds edit boxes, type the lower and upper bounds of the equivalence interval. For example, a hypothesized value of 65 with equivalence bounds of ±5. Alternatively, in the Hypothesized value edit box, type -, and then in the Equivalence bounds edit box, type the lower and upper limits of the equivalence interval. For example, 60 to 70.

6. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

7. Click Calculate.

Related concepts

Parameter equivalence hypothesis test

Related reference

Tests for the central location parameter of a distribution

Discrete distributions

A discrete distribution describes a variable that can only take discrete values (for example, the number of male and females, or the number of people with a specific eye color).

- Frequency distribution
A frequency distribution reduces a large amount of data into a more easily understandable form.

- Inferences about Binomial distribution parameters
Inferences about the parameters of a binomial distribution are made using a random sample of data drawn from the population of interest.

- Inferences about Multinomial distribution parameters
Inferences about the parameters of a multinomial distribution are made using a random sample of data drawn from the population of interest.

Frequency distribution

A frequency distribution reduces a large amount of data into a more easily understandable form.

- Frequency table
A frequency table is a simple table of the frequencies in each class.

- Frequency plot
A frequency plot shows the distribution of a qualitative variable.

- Whole-to-part plot
A whole-to-part plot shows how the parts make up the whole.

Frequency table

A frequency table is a simple table of the frequencies in each class.

There are many ways of expressing the frequencies:

Statistic

Purpose

Frequency

The number of occurrences in the class.

Relative frequency

The frequency divided by the total.

Frequency density

The relative frequency divided by the width of the class interval.
Use with unequal class intervals.

Cumulative frequency

The number of occurrences in the class and all previous classes.

Cumulative relative frequency

The cumulative frequency divided by the total.

Frequency plot

A frequency plot shows the distribution of a qualitative variable.

A frequency plot shows rectangular bars for each class with the height of the bar proportional to the frequency.

Related concepts

Whole-to-part plot

Frequency table

Related tasks

Creating a frequency plot

Whole-to-part plot

A whole-to-part plot shows how the parts make up the whole.

A pie chart is a common representation which shows a circle divided into sectors for each class, with the angle of each sector proportional to the frequency.

A stacked bar plot shows rectangular bars for each class with the size of each bar proportional to the frequency.

The major problem with the pie chart is that it is difficult to judge the angles and areas of the sectors. It is useful if you want to compare a single class relative to the whole. For comparing classes to each other, we recommend the frequency plot or stacked bar plot for greater clarity and ease of interpretation.

Related concepts

Frequency plot

Related tasks

Creating a whole-to-part plot

Creating a frequency plot

Plot a frequency plot to visualize the distribution of a qualitative variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution > Frequency plot, and then click the plot type.
The analysis task pane opens.

3. In the Y drop-down list, select the categorical variable.

4. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

5. Optional: To show the frequency table, select the Frequency table check box.

6. Optional: To vary the bar color by the categorical variable, select the Vary bar color check box.
The categories are assigned default colors. To set specific colors for each category, alongside the Y variable, click the drop-down arrow, and then click Colors / Symbols....

7. Optional: To label the bars with the frequencies, select the Label bars check box.

8. Optional: To overlay the cumulative frequency distribution line, select the Cumulative frequency line check box.

9. Click Calculate.

Related concepts

Frequency plot

Related reference

Dataset layout

Creating a whole-to-part plot

Plot a pie chart or bar chart to visualize the relative frequencies.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution > Whole-to-Part, and then click the plot type.
The analysis task pane opens.

3. In the Y drop-down list, select the categorical variable.

4. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

5. Optional: To show the frequency table, select the Frequency table check box.

6. Optional: To label the bars/sectors with the frequencies, select the Label bars/Label sectors check box.

7. Click Calculate.

Related concepts

Whole-to-part plot

Related reference

Dataset layout

Inferences about Binomial distribution parameters

Inferences about the parameters of a binomial distribution are made using a random sample of data drawn from the population of interest.

A binomial distribution arises when an experiment consists of a fixed number of repeated trials; each trial has two possible outcomes; the probability of the outcome is the same for each trial; and the trials are independent, that is, the outcome of one trial does not affect the outcome of other trials.

- Binomial distribution parameter estimate
A parameter estimate is either a point or interval estimate of an unknown population parameter.

- Binomial distribution parameter hypothesis test
A hypothesis test formally tests if a population parameter is equal to a hypothesized value. For a binomial distribution, the parameter is the probability of success, commonly referred to as the proportion with the outcome of interest.

Binomial distribution parameter estimate

A parameter estimate is either a point or interval estimate of an unknown population parameter.

A point estimate is a single value that is the best estimate of the true unknown parameter; a confidence interval is a range of values and indicates the uncertainty of the estimate.

Related concepts

Point and interval estimation

Related reference

Estimators for the parameter of a Binomial distribution

Estimators for the parameter of a Binomial distribution

Estimators for the binomial distribution parameter and their properties and assumptions.

Estimator

Purpose

Proportion

Estimate the population proportion of occurrences of the outcome of interest using the sample proportion estimator.

Odds

Estimate the population odds of the outcome of interest occurring using the sample odds estimator.
Odds is an expression of the relative probabilities in favor of an event.

Related concepts

Binomial distribution parameter estimate

Binomial distribution parameter hypothesis test

A hypothesis test formally tests if a population parameter is equal to a hypothesized value. For a binomial distribution, the parameter is the probability of success, commonly referred to as the proportion with the outcome of interest.

The null hypothesis states that the proportion is equal to the hypothesized value, against the alternative hypothesis that it is not equal to (or less than, or greater than) the hypothesized value. When the test p-value is small, you can reject the null hypothesis and conclude the sample is not from a population with the proportion equal to the hypothesized value.

Related concepts

Hypothesis testing

Related tasks

Testing Binomial distribution parameters

Related reference

Tests for the parameter of a Binomial distribution

Tests for the parameter of a Binomial distribution

Tests for the parameter of a binomial distribution and their properties and assumptions.

Test

Purpose

Binomial exact

Test if the proportion with the outcome of interest is equal to a hypothesized value.
Uses the binomial distribution and computes an exact p-value. The test is conservative, that is, the type I error is guaranteed to be less than or equal to the desired significance level. Recommended for small sample sizes.

Score Z

Test if the proportion with the outcome of interest is equal to a hypothesized value.
Uses the score statistic and computes an asymptotic p-value. Equivalent to Pearson's X² test. Recommended for general use.

Related concepts

Binomial distribution parameter hypothesis test

Related tasks

Testing Binomial distribution parameters

Testing Binomial distribution parameters

Test if the parameter of a binomial distribution is equal to the hypothesized value.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution, and then click the hypothesis test.
The analysis task pane opens.

3. In the Y drop-down list, select the categorical variable.

4. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

5. In the Hypotheses drop-down list, select the null and alternative hypothesis.

6. In the Hypothesized value edit box, type the expected value of the parameter under the null hypothesis.

7. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

8. Click Calculate.

Related concepts

Binomial distribution parameter hypothesis test

Related reference

Dataset layout

Tests for the parameter of a Binomial distribution

Inferences about Multinomial distribution parameters

Inferences about the parameters of a multinomial distribution are made using a random sample of data drawn from the population of interest.

A multinomial distribution arises when an experiment consists of a fixed number of repeated trials; each trial has a discrete number of possible outcomes; the probability that a particular outcome occurs is the same for each trial; and the trials are independent, that is, the outcome of one trial does not affect the outcome of other trials.

- Multinomial distribution parameters hypothesis test
A hypothesis test formally tests if the population parameters are different from the hypothesized values. For a multinomial distribution, the parameters are the proportions of occurrence of each outcome.

Multinomial distribution parameters hypothesis test

A hypothesis test formally tests if the population parameters are different from the hypothesized values. For a multinomial distribution, the parameters are the proportions of occurrence of each outcome.

The null hypothesis states that the proportions equal the hypothesized values, against the alternative hypothesis that at least one of the proportions is not equal to its hypothesized value. When the test p-value is small, you can reject the null hypothesis and conclude that at least one proportion is not equal to its hypothesized value.

The test is an omnibus test and does not tell you which proportions differ from the hypothesized values.

Related concepts

Hypothesis testing

Related tasks

Testing Multinomial distribution parameters

Related reference

Tests for the parameters of a Multinomial distribution

Tests for the parameters of a Multinomial distribution

Tests for the parameters of a multinomial distribution and their properties and assumptions.

Test

Purpose

Pearson X²

Tests if the proportions are equal to the hypothesized values.
Uses the score statistic and computes an asymptotic p-value.

Likelihood ratio G²

Tests if the proportions are equal to the hypothesized values.
Uses the likelihood ratio statistic and computes an asymptotic p-value.

Pearson X² usually converges to the chi-squared distribution more quickly than G². The likelihood ratio test is commonly used in statistical modeling as the G² statistic is easier to compare between different models.

Related concepts

Multinomial distribution parameters hypothesis test

Related tasks

Testing Multinomial distribution parameters

Testing Multinomial distribution parameters

Test if the parameters of a multinomial distribution are equal to the hypothesized values.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Distribution, and then click the hypothesis test.
The analysis task pane opens.

3. In the Y drop-down list, select the categorical variable.

4. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

5. In the Hypotheses drop-down list, select the null and alternative hypothesis.

6. In the Hypothesized values group, select:

- Are all equal to set the hypothesized values equal for each category.

- Are equal to specified, and then in the grid, under the Value column alongside each category, type values as either probabilities/proportions or number of occurrences.

7. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

8. Click Calculate.

Related concepts

Multinomial distribution parameters hypothesis test

Related reference

Dataset layout

Tests for the parameters of a Multinomial distribution

Study design

Distribution analysis study requirements and dataset layout.

Requirements

- A categorical or quantitative variable.

Dataset layout

Use a column for each variable (Height, Eye color); each row has the values of the variables for a case (Subject).

Subject (optional)

Height

Eye color

1

175

Blue

2

180

Blue

3

160

Hazel

4

190

Green

5

180

Green

6

150

Brown

7

140

Blue

8

160

Brown

9

165

Green

10

180

Hazel

…

…

…

Frequency form dataset layout

Use a column for the variable (Eye color) and a column for the number of cases (Frequency); each row has the values of the variables and the frequency count.

Eye color

Frequency

Brown

221

Blue

215

Hazel

93

Green

64

Chapter 10. Compare groups

Compare Groups examines independent samples and makes inferences about the differences between them.

Independent samples occur when observations are made on different sets of items or subjects. If the values in one sample do not tell you anything about the values in the other sample, then the samples are independent. If the knowing the values in one sample could tell you something about the values in the other sample, then the samples are related.

- Side-by-side univariate plots
Side-by-side univariate plots summarize the distribution of data stratified into groups.

- Equality of means/medians hypothesis test
An equality hypothesis test formally tests if two or more population means/medians are different.

- Equivalence of means hypothesis test
An equivalence hypothesis test formally tests if two population means are equivalent, that is, practically the same.

- Difference between means/medians effect size
An effect size estimates the magnitude of the difference between two means/medians.

- Multiple comparisons
Multiple comparisons make simultaneous inferences about a set of parameters.

- Mean-Mean scatter plot
A mean-mean scatter plot shows a 2-dimensional representation of the differences between many means.

- Homogeneity of variance hypothesis test
A homogeneity hypothesis test formally tests if the populations have equal variances.

Calculating univariate descriptive statistics, by group

Use univariate descriptive statistics to describe a quantitative variable that is classified by a factor variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups, and then click the statistics to show:

Option

Description

Mean and Moments

Show mean, variance, standard deviation.

Median and Quartiles

Show minimum, maximum, median, and quartiles.

The analysis task pane opens.

3. In the Y drop-down list, select the quantitative response variable.

4. In the X drop-down list, select the categorical factor variable identifying the groups.

5. Optional: To customize the statistics to show, click Customize... and then select or clear the appropriate check boxes. To save the options as the defaults to use for future analyses, click Save as Defaults.

6. Click Calculate.

Related concepts

Univariate descriptive statistics

Related reference

Dataset layout

Side-by-side univariate plots

Side-by-side univariate plots summarize the distribution of data stratified into groups.

Related tasks

Creating side-by-side univariate plots

Creating side-by-side univariate plots

Summarize the distribution of a single quantitative variable stratified into groups.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups > Univariate, and then click the plot type.
The analysis task pane opens.

3. In the Y drop-down list, select the quantitative response variable.

4. In the X drop-down list, select the categorical factor variable identifying the groups.

5. Optional: To overlay more than one plot type on the same chart, on the Analyse-it ribbon tab, in the Compare Groups group, click Univariate, and then click an additional plot type.

6. Optional: In the analysis task pane, customize the options for the plot.

7. Click Calculate.

Related concepts

Side-by-side univariate plots

Related reference

Dataset layout

Equality of means/medians hypothesis test

An equality hypothesis test formally tests if two or more population means/medians are different.

The hypotheses to test depends on the number of samples:

- For two samples, the null hypothesis states that the difference between the mean/medians of the populations is equal to a hypothesized value (0 indicating no difference), against the alternative hypothesis that it is not equal to (or less than, or greater than) the hypothesized value.

- For more than two samples, the null hypothesis states that the means/medians of the populations are equal, against the alternative hypothesis that at least one population mean/median is different.

When the test p-value is small, you can reject the null hypothesis and conclude that the populations differ in means/medians.

Tests for more than two samples are omnibus tests and do not tell you which groups differ from each other. You should use multiple comparisons to make these inferences.

Related concepts

Hypothesis testing

Multiple comparisons

Related tasks

Testing equality of means/medians

Related reference

Tests for means/medians

Equivalence of means hypothesis test

An equivalence hypothesis test formally tests if two population means are equivalent, that is, practically the same.

An equality hypothesis test can never prove that the means are equal, it can only ever disprove the null hypothesis of equality. It is therefore of interest when comparing say a new treatment against a placebo, where the null hypothesis (assumption of what is true without evidence to the contrary) is that the treatment has no effect, and you want to prove the treatment produces a useful effect. By contrast, an equivalence hypothesis test is of interest when comparing say a generic treatment to an existing treatment where the aim is to prove that they are equivalent, that is the difference is less than some small negligible effect size. A equivalence hypothesis test therefore constructs the null hypothesis of non-equivalence and the goal is to prove the means are equivalent.

The null hypothesis states that the means are not equivalent, against the alternative hypothesis that the difference between the means is within the bounds of the equivalence interval, that is, the effect size is less than some small difference that is considered practically zero. The hypothesis is tested as a composite of two one-sided t-tests (TOST), H0_1 tests the hypothesis that mean difference is less than the lower bound of the equivalence interval, test H0_2 that the mean difference is greater than the upper bounds of the equivalence interval. The p-value is the greater of the two one sided t-test p-values. When the test p-value is small, you can reject the null hypothesis and conclude the samples are from populations with practically equivalent means.

Related tasks

Testing equivalence of means

Related reference

Tests for means/medians

Tests for means/medians

Tests for the equality/equivalence of means/medians of independent samples and their properties and assumptions.

Test

Purpose

Z

Test if the difference between means is equal to a hypothesized value when the population standard deviation is known.

Student's t

Test if the difference between means is equal to a hypothesized value.
Assumes the populations are normally distributed. Due to the central limit theorem, the test may still be useful when this assumption is not true if the sample sizes are equal, moderate size, and the distributions have a similar shape. However, in this situation the Wilcoxon-Mann-Whitney test may be more powerful.

Assumes the population variances are equal. This assumption can be tested using the Levene test. The test may still be useful when this assumption is not true if the sample sizes are equal. However, in this situation, the Welch t-test may be preferred.

Welch t

Test if the difference between means is equal to a hypothesized value.
Assumes the populations are normally distributed. Due to the central limit theorem, the test may still be useful when this assumption is not true if the sample sizes are equal, moderate size, and the distributions have a similar shape. However, in this situation the Wilcoxon-Mann-Whitney test may be more powerful.

Does not assume the population variances are equal.

TOST (two-one-sided t-tests)

Test if the means are equivalent.
Assumes the populations are normally distributed. Due to the central limit theorem, the test may still be useful when this assumption is not true if the sample sizes are equal, moderate size, and the distributions have a similar shape.

Note: The TOST can be performed using either a Student's t-test or Welch t-test.

ANOVA

Test if two or means are equal.
Assumes the populations are normally distributed. Due to the central limit theorem, the test may still be useful when this assumption is not true if the sample sizes are equal and moderate size. However, in this situation the Kruskal-Wallis test is may be more powerful.

Assumes the population variances are equal. This assumption can be tested using the Levene test. The test may still be useful when this assumption is not true if the sample sizes are equal. However, in this situation, the Welch ANOVA may be preferred.

Welch ANOVA

Test if two or more means are equal.
Assumes the populations are normally distributed. Due to the central limit theorem, the test may still be useful when this assumption is not true when the sample sizes are equal and moderate size. The Kruskal-Wallis test may be preferable as it is more powerful than Welch's ANOVA.

Does not assume the population variances are equal.

Wilcoxon-Mann-Whitney

Test if there is a shift in location.
When the population distributions are identically shaped, except for a possible shift in central location, the hypotheses can be stated in terms of a difference between means/medians.

When the population distributions are not identically shaped, the hypotheses can be stated as a test of whether the samples come from populations such that the probability is 0.5 that a random observation from one group is greater than a random observation from another group.

Kruskal-Wallis

Test if two or more medians are equal.
Assumes the population distributions are identically shaped, except for a possible shift in central location.

Related concepts

Equality of means/medians hypothesis test

Central limit theorem and the normality assumption

Equivalence of means hypothesis test

Related tasks

Testing equality of means/medians

Testing equivalence of means

Testing equality of means/medians

Test if there is a difference between the means/medians of two or more independent samples.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups, and then click the hypothesis test.
The analysis task pane opens.

3. In the Y drop-down list, select the quantitative response variable.

4. In the X drop-down list, select the categorical factor variable identifying the groups.

5. If performing a 2 sample hypothesis test:

1. In the Hypotheses drop-down list, select the null and directional alternative hypotheses.

2. In the Hypothesized difference edit box, type the expected difference under the null hypothesis.

6. If performing a k sample hypothesis test:

1. In the Hypothesis drop-down list, select the null and alternative hypotheses.

7. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

8. Click Calculate.

Related concepts

Equality of means/medians hypothesis test

Related reference

Dataset layout

Tests for means/medians

Testing equivalence of means

Test if there is equivalence of the means of two independent samples.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups, and then click TOST hypothesis test.
The analysis task pane opens.

3. In the Y drop-down list, select the quantitative response variable.

4. In the X drop-down list, select the categorical factor variable identifying the groups.

5. In the Test drop-down list, select either Student's t or Welch t depending on the homogeneity of variances.
The TOST equivalence test is actually specified through the Hypotheses drop-down list, the Test drop-down list indicates the type of t-test to use.

6. In the Equivalence interval edit boxes, type the lower and upper bounds of the equivalence interval. For example, type-5 to +5 if differences between the means of ±5 is the effect size considered to be practically the same.

7. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

8. Click Calculate.

Related concepts

Equivalence of means hypothesis test

Related reference

Tests for means/medians

Difference between means/medians effect size

An effect size estimates the magnitude of the difference between two means/medians.

The term effect size relates to both unstandardized measures (for example, the difference between group means) and standardized measures (such as Cohen's d; the standardized difference between group means). Standardized measures are more appropriate than unstandardized measures when the metrics of the variables do not have intrinsic meaning, when combining results from multiple studies, or when comparing results from studies with different measurement scales.

A point estimate is a single value that is the best estimate of the true unknown parameter; a confidence interval is a range of values and indicates the uncertainty of the estimate.

Related concepts

Point and interval estimation

Related tasks

Estimating the difference between means/medians

Related reference

Estimators for the difference in means/medians

Estimators for the difference in means/medians

Estimators for the difference in means/medians of independent samples and their properties and assumptions.

Estimator

Purpose

Mean difference

Estimate the difference between the means.

Standardized mean difference

Estimate the standardized difference between the means.
Cohen's d is the most popular estimator using the difference between the means divided by the pooled sample standard deviation. Cohen's d is a biased estimator of the population standardized mean difference (although the bias is very small and disappears for moderate to large samples) whereas Hedge's g applies an unbiasing constant to correct for the bias.

Hodges-Lehmann location shift

Estimate the shift in location.
A shift in location is equivalent to a difference between means/medians when the distributions are identically shaped.

Related concepts

Difference between means/medians effect size

Related tasks

Estimating the difference between means/medians

Estimating the difference between means/medians

Estimate the difference between the means/medians of 2 independent samples.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups, and then click the effect size estimator.
The analysis task pane opens.

3. In the Y drop-down list, select the quantitative response variable.

4. In the X drop-down list, select the categorical factor variable identifying the groups.

5. In the Difference drop-down list, select the direction of the difference.

6. In the Confidence interval edit box, type the confidence level as a percentage, or type - to suppress the confidence interval, and then in the drop-down list, select the confidence bounds.

7. In the Method drop-down list, select the interval estimator.

8. Click Calculate.

Related concepts

Difference between means/medians effect size

Related reference

Dataset layout

Estimators for the difference in means/medians

Multiple comparisons

Multiple comparisons make simultaneous inferences about a set of parameters.

When making inferences about more than one parameter (such as comparing many means, or the differences between many means), you must use multiple comparison procedures to make inferences about the parameters of interest. The problem when making multiple comparisons using individual tests such as Student's t-test applied to each comparison is the chance of a type I error increases with the number of comparisons. If you use a 5% significance level with a hypothesis test to decide if two groups are significantly different, there is a 5% probability of observing a significant difference that is simply due to chance (a type I error). If you made 20 such comparisons, the probability that one or more of the comparisons is statistically significant simply due to chance increases to 64%. With 50 comparisons, the chance increases to 92%. Another problem is the dependencies among the parameters of interest also alter the significance level. Therefore, you must use multiple comparison procedures to maintain the simultaneous probability close to the nominal significance level (typically 5%).

Multiple comparison procedures are classified by the strength of inference that can be made and the error rate controlled. A test of homogeneity controls the probability of falsely declaring any pair to be different when in fact all are the same. A stronger level of inference is confident inequalities and confident directions which control the probability of falsely declaring any pair to be different regardless of the values of the others. An even stronger level is a set of simultaneous confidence intervals that guarantees that the simultaneous coverage probability of the intervals is at least 100(1-alpha)% and also have the advantage of quantifying the possible effect size rather than producing just a p-value. A higher strength inference can be used to make an inference of a lesser strength but not vice-versa. Therefore, a confidence interval can be used to perform a confidence directions/inequalities inference, but a test of homogeneity cannot make a confidence direction/inequalities inference.

The most well known multiple comparison procedures, Bonferroni and Šidák, are not multiple comparison procedures per se. Rather they are an inequality useful in producing easy to compute multiple comparison methods of various types. In most scenarios, there are more powerful procedures available. A useful application of Bonferroni inequality is when there are a small number of pre-planned comparisons. In these cases, you can use the standard hypothesis test or confidence interval with the significance level (alpha) set to the Bonferroni inequality (alpha divided by the number of comparisons).

A side effect of maintaining the significance level is a lowering of the power of the test. Different procedures have been developed to maintain the power as high as possible depending on the strength of inference required and the number of comparisons to be made. All contrasts comparisons allow for any possible contrast; all pairs forms the k*(k-1)/2 pairwise contrasts, whereas with best forms k contrasts each with the best of the others, and against control forms k-1 contrasts each against the control group. You should choose the appropriate contrasts of interest before you perform the analysis, if you decide after inspecting the data, then you should only use all contrasts comparison procedures.

Related concepts

Mean-Mean scatter plot

Point and interval estimation

Hypothesis testing

Related tasks

Comparing multiple means/medians

Related reference

Multiple comparison procedures

Mean-Mean scatter plot

A mean-mean scatter plot shows a 2-dimensional representation of the differences between many means.

The mean-mean scatter plot shows the mean of a group on the horizontal axis against the mean of the other group on the vertical axis with a dot at the intersection. A vector centered at the intersection with a slope of -1 and a length proportional to the width of the confidence interval represents the confidence interval. A gray identity line represents equality of means; that is the difference is equal to zero. If the vector does not cross the identity line, you can conclude there is a significant difference between the means.

To make interpretation easier, a 45-degree rotated version of the plot shows the difference between means and its confidence interval on the horizontal axis against average of the means on the vertical axis.

Related concepts

Multiple comparisons

Related tasks

Comparing multiple means/medians

Multiple comparison procedures

Methods of controlling the Type I error and dependencies between parameters when making multiple comparisons between many means/medians of independent samples.

Procedure

Purpose

Student's t (Fisher's LSD)

Compare the means of each pair of groups using the Student's t method.
When making all pairwise comparisons this procedure is also known as unprotected Fisher's LSD, or when only performed following significant ANOVA F -test known as protected Fisher's LSD.

Control the type I error rate is for each contrast.

Wilcoxon-Mann-Whitney

Compare the median/means each pair of groups using the Wilcoxon nonparametric method.
Control the type I error rate for each contrast.

Tukey-Kramer

Compare the means of all pairs of groups using the Tukey-Kramer method.
Controls the error rate simultaneously for all k(k+1)/2 contrasts.

Steel-Dwass-Critchlow-Fligner

Compare the median/means of all pairs of groups using the Steel-Dwass-Critchlow-Fligner pairwise ranking nonparametric method.
Controls the error rate simultaneously for all k(k+1)/2 contrasts.

Hsu

Compare the means of all groups against the best of the other groups using the Hsu method.
Controls the error rate simultaneously for all k contrasts.

Dunnett

Compare the means of all groups against a control using the Dunnett method.
Controls the error rate simultaneously for all k-1 contrasts.

Steel

Compare the medians/means of all groups against a control using the Steel pairwise ranking nonparametric method.
Controls the error rate simultaneously for all k-1 comparisons.

Scheffé

Compare the means of all groups against all other groups using the Scheffé F method.
Controls the error rate simultaneously for all possible contrasts.

Student-Newman-Keuls (SNK)

Not implemented.
As discussed in Hsu (1996) it is not a confident inequalities method and cannot be recommended.

Duncan

Not implemented.
As discussed in Hsu (1996) it is not a confident inequalities method and cannot be recommended.

Dunn

Not implemented in Analyse-it version 3 onwards. Implemented in version 1.
A joint-ranking nonparametric method but as discussed in Hsu (1996) it is not a confident inequalities method and cannot be recommended. Use the Dwass-Steel-Critchlow-Fligner or Steel test instead.

Conover-Iman

Not implemented in Analyse-it version 3 onwards. Implemented in version 2.
As discussed in Hsu (1996) applying parametric methods to the rank-transformed data may not produce a confident inequalities method even if the parametric equivalent it mimics is. Use the Dwass-Steel-Critchlow-Fligner or Steel test instead.

Bonferroni

Not a multiple comparisons method. It is an inequality useful in producing easy to compute multiple comparisons. In most scenarios, there are more powerful procedures such as Tukey, Dunnett, Hsu.
A useful application of Bonferroni inequality is when there are a small number of pre-planned comparisons. In this case, use the Student's t (LSD) method with the significance level (alpha) set to the Bonferroni inequality (alpha divided by the number of comparisons). In this scenario, it is usually less conservative than using Scheffé all contrast comparisons.

Related concepts

Multiple comparisons

Related tasks

Comparing multiple means/medians

Related information

Hsu, J. C. (1996). Multiple Comparisons: Theory and Methods. CRC Press.

Comparing multiple means/medians

Compare the means/medians of many independent samples.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups, and then click the multiple comparisons procedure.
The analysis task pane opens.

3. In the Y drop-down list, select the quantitative response variable.

4. In the X drop-down list, select the categorical factor variable identifying the groups.

5. In the Type drop-down list, select the comparisons, and then:

- If comparing with best, in the Best drop-down list, select the "best" group.

- If comparing against a control, in the Control drop-down list, select the control group.

6. Optional: To sort the contrasts by the magnitude of the difference, select the Ordered check box.

7. Optional: In the Method drop-down list, select the multiple comparisons procedure, and in the Confidence interval edit box, type the confidence level as a percentage, or type - to suppress the confidence interval, and then in the Confidence bounds drop-down list, select the limits.

8. Optional: To perform a hypothesis test to test if the difference is equal to 0, select the Hypothesis test check box, and then to compare the p-value, in the Significance level edit box, type the predefined significance level.

9. Optional: To show a mean-mean scatter plot, select the Mean-Mean scatter plot check box.

10. Click Calculate.

Related concepts

Multiple comparisons

Mean-Mean scatter plot

Related reference

Dataset layout

Multiple comparison procedures

Homogeneity of variance hypothesis test

A homogeneity hypothesis test formally tests if the populations have equal variances.

Many statistical hypothesis tests and estimators of effect size assume that the variances of the populations are equal. This assumption allows the variances of each group to be pooled together to provide a better estimate of the population variance. A better estimate of the variance increases the statistical power of the test meaning you can use a smaller sample size to detect the same difference, or detect smaller differences and make sharper inferences with the same sample size.

The hypotheses to test depends on the number of samples:

- For two samples, the null hypothesis states that the ratio of the variances of the populations is equal to a hypothesized value (usually 1 indicating equality), against the alternative hypothesis that it is not equal to (or less than, or greater than) the hypothesized value.

- For more than two samples, the null hypothesis states that the variances of the populations are equal, against the alternative hypothesis that at least one population variance is different.

When the test p-value is small, you can reject the null hypothesis and conclude that the populations differ in variance.

Related concepts

Hypothesis testing

Related tasks

Testing homogeneity of variance

Related reference

Tests for homogeneity of variance

Tests for homogeneity of variance

Tests for the homogeneity of variance of independent samples and their properties and assumptions.

Test

Purpose

F

Test if the ratio of the variances is equal to a hypothesized value.
Assumes the populations are normally distributed and is extremely sensitive to departures from normality. The Levene and Brown-Forsythe tests are more robust against violations of the normality assumption than the F-test.

Bartlett

Test if the variances are equal.
Assumes the populations are normally distributed and is extremely sensitive to departures from normality. The Levene and Brown-Forsythe tests are more robust against violations of the normality assumption than Bartlett's test.

Levene

Test if the variances are equal.
Most powerful test for symmetric, moderate-tailed, distributions.

Brown-Forsythe

Test if the variances are equal.
Robust against many types of non-normality.

Related concepts

Homogeneity of variance hypothesis test

Related tasks

Testing homogeneity of variance

Testing homogeneity of variance

Test if the variances of two or more independent samples are equal.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups, and then click the hypothesis test.
The analysis task pane opens.

3. In the Y drop-down list, select the quantitative response variable.

4. In the X drop-down list, select the categorical factor variable identifying the groups.

5. In the Hypotheses drop-down list, select the null and alternative hypothesis.

6. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

7. Click Calculate.

Related concepts

Homogeneity of variance hypothesis test

Related reference

Dataset layout

Tests for homogeneity of variance

Study design

Compare groups analysis study requirements and dataset layout.

Requirements

- A quantitative response variable.

- A categorical factor variable with 2 or more levels.

Dataset layout

Use a column for the response variable (Height) and a column for the factor variable (Sex); each row has the values of the variables for a case (Subject).

Subject (optional)

Sex

Height

1

Male

175

2

Male

180

3

Male

160

4

Male

190

5

Male

180

6

Female

150

7

Female

140

8

Female

160

9

Female

165

10

Female

180

…

…

…

Note: Do not split groups into separate columns. For example, you should not record the heights of males in one column and heights of females in another.

Chapter 11. Compare pairs

Compare Pairs examines related samples and makes inferences about the differences between them.

Related samples occur when observations are made on the same set of items or subjects at different times, or when another form of matching has occurred. If the knowing the values in one sample could tell you something about the values in the other sample, then the samples are related.

There are a few different study designs that produce related data. A paired study design takes individual observations from a pair of related subjects. A repeat measures study design takes multiple observations on the same subject. A matched pair study design takes individual observations on multiple subjects that are matched on other covariates. The purpose of matching similar subjects is often to reduce or eliminate the effects of a confounding factor.

- Difference plot
A difference plot shows the differences between two observations on the same sampling unit.

- Equality of means/medians hypothesis test
An equality hypothesis test formally tests if two or more population means/medians are different.

- Equivalence of means hypothesis test
An equivalence hypothesis test formally tests if two population means are equivalent, that is, practically the same.

- Difference between means/medians effect size
An effect size estimates the magnitude of the difference between two means/medians.

Difference plot

A difference plot shows the differences between two observations on the same sampling unit.

The difference plot shows the difference between two observations on the vertical axis against the average of the two observations on the horizontal axis. A gray identity line represents equality; no difference.

If the second observation is always greater than the first the points lie above the line of equality, or vice-versa. If differences are not related to the magnitude the points will form a horizontal band. If the points form an increasing, decreasing, or non-constant width band, then the variance is not constant.

It is common to combine the difference plot with a histogram and a normality plot of the differences to check if the differences are normally distributed, which is an assumption of some statistical tests and estimators.

Related tasks

Creating a Tukey mean-difference plot

Creating a Tukey mean-difference plot

Summarize the differences between two related observations.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Pairs, and then click Difference.
The analysis task pane opens.

3. If the data consists of paired/repeated measurements in separate variables:

1. In the Model drop-down menu, select Paired/Repeated.

2. In the Y list, select the quantitative variables.

4. If the data consist of matched pairs with a separate response variable, factor variable, and blocking variable:

1. In the Model drop-down menu, select Matched.

2. In the Y drop-down list, select the quantitative response variable.

3. In the X (factor) drop-down list, select the categorical factor variable identifying the groups.

4. In the Block drop-down list, select the blocking variable identifying the matching.

5. Optional: To show a histogram of the distribution of the differences, select the Histogram check box.

6. Optional: To show a normality plot of the differences, select the Normal plot check box.

7. Click Calculate.

Related concepts

Difference plot

Related reference

Dataset layout

Equality of means/medians hypothesis test

An equality hypothesis test formally tests if two or more population means/medians are different.

Inferences about related samples are complicated by the fact that the observations are correlated. Therefore the tests for independent samples are of no use. Instead, tests for related samples focus on the differences within each sampling unit.

The hypotheses to test depends on the number of samples:

- For two samples, the null hypothesis states that the difference between the mean/medians of the populations is equal to a hypothesized value (0 indicating no difference), against the alternative hypothesis that it is not equal to (or less than, or greater than) the hypothesized value.

- For more than two samples, the null hypothesis states that the means/medians of the populations are equal, against the alternative hypothesis that at least one population mean/median is different.

When the test p-value is small, you can reject the null hypothesis and conclude that the populations differ in means/medians.

Related concepts

Hypothesis testing

Related tasks

Testing equality of means/medians

Related reference

Tests for means/medians

Equivalence of means hypothesis test

An equivalence hypothesis test formally tests if two population means are equivalent, that is, practically the same.

An equality hypothesis test can never prove that the means are equal, it can only ever disprove the null hypothesis of equality. It is therefore of interest when comparing say a new treatment against a placebo, where the null hypothesis (assumption of what is true without evidence to the contrary) is that the treatment has no effect, and you want to prove the treatment produces a useful effect. By contrast, an equivalence hypothesis test is of interest when comparing say a generic treatment to an existing treatment where the aim is to prove that they are equivalent, that is the difference is less than some small negligible effect size. A equivalence hypothesis test therefore constructs the null hypothesis of non-equivalence and the goal is to prove the means are equivalent.

The null hypothesis states that the means are not equivalent, against the alternative hypothesis that the difference between the means is within the bounds of the equivalence interval, that is, the effect size is less than some small difference that is considered practically zero. The hypothesis is tested as a composite of two one-sided t-tests (TOST), H0_1 tests the hypothesis that mean difference is less than the lower bound of the equivalence interval, test H0_2 that the mean difference is greater than the upper bounds of the equivalence interval. The p-value is the greater of the two one-sided t-test p-values. When the test p-value is small, you can reject the null hypothesis and conclude the samples are from populations with practically equivalent means.

Related tasks

Testing equivalence of means

Related reference

Tests for means/medians

Tests for means/medians

Tests for the equality of means/medians of related samples and their properties and assumptions.

Test

Purpose

Z

Test if the difference between means is equal to a hypothesized value when the population standard deviation is known.
Assumes the population differences are normally distributed. Due to the central limit theorem, the test may still be useful when this assumption is not true if the sample size is moderate. However, in this case, the Wilcoxon test may be more powerful.

Student's t

Test if the difference between means is equal to a hypothesized value.
Assumes the population differences are normally distributed. Due to the central limit theorem, the test may still be useful when this assumption is not true if the sample size is moderate. However, in this case, the Wilcoxon test may be more powerful.

Wilcoxon

Test if there is a shift in location equal to the hypothesized value.
Under the assumption that the population distribution of the differences is symmetric, the hypotheses can be stated in terms of a difference between means/medians.

Under the less strict hypotheses, requiring no distributional assumptions, the hypotheses can be stated as the probability that the sum of a randomly chosen pair of differences exceeds zero is 0.5.

Sign

Test if the median of the differences is equal to a hypothesized value.
Under the more general hypotheses, tests if given a random pair of observations (xi, yi), that xi and yi are equally likely to be larger than the other.

Has few assumptions, but lacks power compared to the Wilcoxon and Student's t test.

TOST (two-one-sided t-tests)

Test if the means are equivalent.
Assumes the populations are normally distributed. Due to the central limit theorem, the test may still be useful when this assumption is not true if the sample sizes are equal, moderate size, and the distributions have a similar shape.

ANOVA

Test if the two or means are equal.
Assumes the populations are normally distributed. Due to the central limit theorem, the test may still be useful when the assumption is violated if the sample sizes are equal and moderate size. However, in this situation the Friedman test is may be more powerful.

Friedman

Test if two or medians are equal.
Has few assumptions, and is equivalent to a two-sided Sign test in the case of two samples.

Related concepts

Equality of means/medians hypothesis test

Central limit theorem and the normality assumption

Equivalence of means hypothesis test

Related tasks

Testing equality of means/medians

Testing equivalence of means

Testing equality of means/medians

Test if there is a difference between the means/medians of two or more related samples.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Pairs, and then click the hypothesis test.
The analysis task pane opens.

3. If the data consists of paired/repeated measurements in separate variables:

1. In the Model drop-down menu, select Paired/Repeated.

2. In the Y list, select the quantitative variables.

4. If the data consist of matched pairs with a separate response variable, factor variable, and blocking variable:

1. In the Model drop-down menu, select Matched.

2. In the Y drop-down list, select the quantitative response variable.

3. In the X (factor) drop-down list, select the categorical factor variable identifying the groups.

4. In the Block drop-down list, select the blocking variable identifying the matching.

5. If performing a 2 sample hypothesis test:

1. In the Hypotheses drop-down list, select the null and directional alternative hypotheses.

2. In the Hypothesized difference edit box, type the expected difference under the null hypothesis.

6. If performing a k sample hypothesis test:

1. In the Hypothesis drop-down list, select the null and alternative hypotheses.

7. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

8. Click Calculate.

Related concepts

Equality of means/medians hypothesis test

Related reference

Dataset layout

Tests for means/medians

Testing equivalence of means

Test if there is equivalence of means of two related samples.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Pairs, and then click TOST hypothesis test.
The analysis task pane opens.

3. If the data consists of paired/repeated measurements in separate variables:

1. In the Model drop-down menu, select Paired/Repeated.

2. In the Y list, select the quantitative variables.

4. If the data consist of matched pairs with a separate response variable, factor variable, and blocking variable:

1. In the Model drop-down menu, select Matched.

2. In the Y drop-down list, select the quantitative response variable.

3. In the X (factor) drop-down list, select the categorical factor variable identifying the groups.

4. In the Block drop-down list, select the blocking variable identifying the matching.

5. In the Equivalence interval edit boxes, type the lower and upper bounds of the equivalence interval. For example, type-5 to +5 if differences between the means of ±5 is the effect size considered to be practically the same.

6. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

7. Click Calculate.

Related concepts

Equivalence of means hypothesis test

Related reference

Tests for means/medians

Difference between means/medians effect size

An effect size estimates the magnitude of the difference between two means/medians.

The term effect size relates to both unstandardized measures (for example, the difference between group means) and standardized measures (such as Cohen's d; the standardized difference between group means). Standardized measures are more appropriate than unstandardized measures when the metrics of the variables do not have intrinsic meaning, when combining results from multiple studies, or when comparing results from studies with different measurement scales.

A point estimate is a single value that is the best estimate of the true unknown parameter; a confidence interval is a range of values and indicates the uncertainty of the estimate.

Related concepts

Point and interval estimation

Related tasks

Estimating the difference between means/medians

Related reference

Estimators for the difference in means/medians

Estimators for the difference in means/medians

Estimators for the difference in means/medians of related samples and their properties and assumptions.

Estimator

Purpose

Mean difference

Estimate the difference between the means.

Standardized mean difference

Estimate the standardized difference between the means.
Cohen's d is the most popular estimator using the difference between the means divided by the pooled sample standard deviation. Cohen's d is a biased estimator of the population standardized mean difference (although the bias is very small and disappears for moderate to large samples) whereas Hedge's g applies an unbiasing constant to correct for the bias.

Median difference

Estimate the median of the differences.

Hodges-Lehmann location shift

Estimate the shift in location.
A shift in location is equivalent to a difference between means/medians when the distribution of the differences is symmetric. Symmetry is quite often inherently satisfied for paired data.

Related concepts

Difference between means/medians effect size

Related tasks

Estimating the difference between means/medians

Estimating the difference between means/medians

Estimate the difference between the means/medians of 2 related samples.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Pairs, and then click the effect size estimator.
The analysis task pane opens.

3. If the data consists of paired/repeated measurements in separate variables:

1. In the Model drop-down menu, select Paired/Repeated.

2. In the Y list, select the quantitative variables.

4. If the data consist of matched pairs with a separate response variable, factor variable, and blocking variable:

1. In the Model drop-down menu, select Matched.

2. In the Y drop-down list, select the quantitative response variable.

3. In the X (factor) drop-down list, select the categorical factor variable identifying the groups.

4. In the Block drop-down list, select the blocking variable identifying the matching.

5. In the Difference drop-down list, select the direction of the difference.

6. In the Confidence interval edit box, type the confidence level as a percentage, or type - to suppress the confidence interval, and then in the drop-down list, select the confidence bounds.

7. In the Method drop-down list, select the interval estimator.

8. Click Calculate.

Related concepts

Difference between means/medians effect size

Related reference

Dataset layout

Estimators for the difference in means/medians

Study design

Compare pairs analysis study requirements and dataset layout.

Requirements

- 2 or more repeated measurements on a quantitative response variable.

Paired/Repeated dataset layout

Use a column for each response variable (Blood pressure - Before, Blood pressure - After); each row has the values of the variables for a case (Subject).

Blood pressure

Subject (optional)

Before

After

1

123

124

2

109

97

3

112

113

4

102

105

5

98

95

6

114

119

7

119

114

8

112

114

9

110

121

10

117

118

…

…

…

Matched dataset layout

Use a column the response variable (Blood pressure), a column for the factor variable (Intervention), and a column for the blocking variable (Pair); each row has the values of the variables for a case.

Pair

Intervention

Blood pressure

1

Before

123

2

Before

109

3

Before

112

4

Before

102

5

Before

98

6

Before

114

7

Before

119

8

Before

112

9

Before

110

10

Before

117

…

Before

…

1

After

124

2

After

97

3

After

113

4

After

105

5

After

95

6

After

119

7

After

114

8

After

114

9

After

121

10

After

118

…

After

…

Chapter 12. Contingency tables

Contingency analysis describes and visualizes the distribution of categorical variables, and makes inferences about the equality of proportions, independence of the variables, or agreement between variables.

- Contingency table
A contingency table, also known as a cross-classification table, describes the relationships between two or more categorical variables.

- Grouped frequency plot
A grouped or stacked frequency plot shows the joint or conditional distributions.

- Effect size
An effect size estimates the magnitude of the difference of proportions or the association between two categorical variables.

- Relative risk
Relative risk is the ratio of the probability of the event occurring in one group versus another group.

- Inferences about equality of proportions
Inferences about the proportions in populations are made using random samples of data drawn from each population of interest.

- Inferences about independence
Inferences about the independence between categorical variables are made using a random bivariate sample of data drawn from the population of interest.

- Mosaic plot
A mosaic plot is a visual representation of the association between two variables.

Contingency table

A contingency table, also known as a cross-classification table, describes the relationships between two or more categorical variables.

A table cross-classifying two variables is called a 2-way contingency table and forms a rectangular table with rows for the R categories of the X variable and columns for the C categories of a Y variable. Each intersection is called a cell and represents the possible outcomes. The cells contain the frequency of the joint occurrences of the X, Y outcomes. A contingency table having R rows and C columns is called an R x C table.

A variable having only two categories is called a binary variable. When both variables are binary, the resulting contingency table is a 2 x 2 table. Also, commonly known as a four-fold table because there are four cells.

Smoke

Alcohol consumption

Yes

No

Total

Low

10

80

90

High

50

40

90

Total

60

120

180

A contingency table can summarize three probability distributions – joint, marginal, and conditional.

- The joint distribution describes the proportion of the subjects jointly classified by a category of X and a category of Y. The cells of the contingency table divided by the total provides the joint distribution. The sum of the joint distribution is 1.

- The marginal distributions describe the distribution of the X (row) or Y (column) variable alone. The row and column totals of the contingency table provide the marginal distributions. The sum of a marginal distribution is 1.

- The conditional distributions describe the distribution of one variable given the levels of the other variable. The cells of the contingency table divided by the row or column totals provide the conditional distributions. The sum of a conditional distribution is 1.

When both variables are random, you can describe the data using the joint distribution, the conditional distribution of Y given X, or the conditional distribution of X given Y.

When one variable is and explanatory variable (X, fixed) and the other a response variable (Y, random), the notion of a joint distribution is meaningless, and you should describe the data using the conditional distribution of Y given X. Likewise, if Y is a fixed variable and X random, you should describe the data using the conditional distribution of X given Y.

When the variables are matched-pairs or repeated measurements on the same sampling unit, the table is square R=C, with the same categories on both the rows and columns. For these tables, the cells may exhibit a symmetric pattern about the main diagonal of the table, or the two marginal distributions may differ in some systematic way.

After 6 months

Before

Approve

Disapprove

Total

Approve

794

150

944

Disapprove

86

570

656

Total

880

720

1600

Related tasks

Creating a contingency table

Creating a contingency table (related data)

Creating a contingency table

Cross-classify data into a 2-way contingency table.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups, and then click Contingency table.
The analysis task pane opens.

3. In the Y drop-down list, select the first categorical variable.

4. In the X drop-down list, select the second categorical variable.

5. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

6. Optional: To customize the statistics to show, click Customize... and then select or clear the appropriate check boxes. To save the options as the default to show for all new analyses, click Save as Defaults.

7. Click Calculate.

Related concepts

Contingency table

Related tasks

Creating a contingency table (related data)

Related reference

Dataset layout

Creating a contingency table (related data)

Cross-classify related data into a 2-way contingency table.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Pairs, and then click Contingency Table.
The analysis task pane opens.

3. If the data consists of paired/repeated measurements in separate variables:

1. In the Model drop-down menu, select Paired / Repeated.

2. In the Y lists, select two categorical variables.

4. If the data consist of matched pairs with a separate response variable, factor variable, and blocking variable:

1. In the Model drop-down menu, select Matched.

2. In the Y drop-down list, select the categorical response variable.

3. In the X (factor) drop-down list, select the categorical factor variable identifying the groups.

4. In the Block drop-down list, select the blocking variable identifying the matching.

5. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

6. Optional: To customize the statistics to show, click Customize... and then select or clear the appropriate check boxes. To save the options as the default to show for all new analyses, click Save as Defaults.

7. Click Calculate.

Related concepts

Contingency table

Related tasks

Creating a contingency table

Related reference

Dataset layout

Grouped frequency plot

A grouped or stacked frequency plot shows the joint or conditional distributions.

Effect size

An effect size estimates the magnitude of the difference of proportions or the association between two categorical variables.

The effect size that best describes a 2 x 2 contingency table depends on the study design that produced the data:

- When both variables are random variables, the odds ratio provides the best measure of association between the variables.

- When one variable is an explanatory variable (a fixed variable) and the other a response variable (a random variable), the effect size between the two groups of the response variable can be expressed as the odds ratio, the difference between proportions, or ratio of proportions.

- When the variables are matched-pairs or repeated measurements, the odds ratio or the difference between proportions are appropriate. The ratio of proportions is meaningless in this scenario.

A point estimate is a single value that is the best estimate of the true unknown parameter; a confidence interval is a range of values and indicates the uncertainty of the estimate.

For tables larger than 2 x 2, you must partition the contingency table into a series of 2 x 2 sub-tables to estimate the effect size.

Related concepts

Relative risk

Point and interval estimation

Related tasks

Estimating the odds ratio

Estimating the odds ratio (related data)

Related reference

Estimators

Estimators

Estimators for contingency tables and their properties and assumptions.

Estimator

Purpose

Proportion difference

Estimate the difference of proportions (also known as Risk Difference).

Proportion ratio

Estimate the ratio of proportions (also known as Risk Ratio).

Odds ratio

Estimate the odds ratio.
Values of the odds ratio range from zero to infinity. Values further from 1 in either direction represent a stronger association. The inverse value of an odds ratio represents the same degree of association in opposite directions.

The odds-ratio does not change value when the orientation of a contingency table is reversed so that rows become columns and columns become rows. Therefore it is useful in situations where both variables are random.

Related concepts

Effect size

Relative risk

Related tasks

Estimating the odds ratio

Estimating the odds ratio (related data)

Estimating the odds ratio

Estimate the odds ratio.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups, and then click Odds Ratio.
The analysis task pane opens.

3. In the Y drop-down list, select the first categorical variable.

4. In the X drop-down list, select the second categorical variable.

5. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

6. In the Ratio drop-down list, select the ratio of interest, and in the Event drop-down list, select the outcome of interest.

7. In the Confidence interval edit box, type the confidence level as a percentage, or type - to suppress the confidence interval, and then in the drop-down list, select the confidence bounds.

8. In the Method drop-down list, select the interval estimator.

9. Click Calculate.

Related concepts

Effect size

Related tasks

Estimating the odds ratio (related data)

Related reference

Dataset layout

Estimators

Estimating the odds ratio (related data)

Estimate the odds ratio for related data.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Pairs, and then click Odds Ratio.
The analysis task pane opens.

3. If the data consists of paired/repeated measurements in separate variables:

1. In the Model drop-down menu, select Paired / Repeated.

2. In the Y lists, select two categorical variables.

4. If the data consist of matched pairs with a separate response variable, factor variable, and blocking variable:

1. In the Model drop-down menu, select Matched.

2. In the Y drop-down list, select the categorical response variable.

3. In the X (factor) drop-down list, select the categorical factor variable identifying the groups.

4. In the Block drop-down list, select the blocking variable identifying the matching.

5. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

6. In the Ratio drop-down list, select the ratio of interest, and in the Event drop-down list, select the outcome of interest.

7. In the Confidence interval edit box, type the confidence level as a percentage, or type - to suppress the confidence interval, and then in the drop-down list, select the confidence bounds.

8. In the Method drop-down list, select the interval estimator.

9. Click Calculate.

Related concepts

Effect size

Related tasks

Estimating the odds ratio

Related reference

Dataset layout

Estimators

Relative risk

Relative risk is the ratio of the probability of the event occurring in one group versus another group.

Medical statistics uses categorical data analysis extensively to describe the association between the occurrence of a disease and exposure to a risk factor. In this application, the terminology is often used to frame the statistics in terms of risk. Risk difference is equivalent to a difference between proportions, and a risk ratio is equivalent to a ratio of proportions.

The term relative risk is the ratio of the probability of the event occurring in the exposed group versus a non-exposed group. It is the same as the risk ratio. Although a relative risk is different from odds ratio, in some circumstances, such as the low prevalence of the disease, the odds ratio asymptotically approaches the relative risk and is, therefore, an estimate of the relative risk. It is better to avoid confusion and use the terms risk ratio and odds ratio for the effect size estimates and use the term relative risk for the population parameter.

A prospective cohort study or a clinical trial can estimate the risk ratio or the odds ratio of the occurrence of a disease given exposure to a risk factor. Whereas, a retrospective case-control study can estimate the odds ratio of the risk factor given the disease, which is equivalent to the odds ratio of the occurrence of the disease given the risk factor. A risk ratio is not useful because it refers to the risk factor given the disease, which is not normally of interest. However, as stated above the odds ratio can be used as an estimate of the relative risk in most circumstances.

Practitioners often prefer the risk ratio due to its more direct interpretation. Statisticians tend to prefer the odds ratio as it applies to a wide range of study designs, allowing comparison between different studies and meta-analysis based on many studies.  It also forms the basis of logistic regression.

Related concepts

Effect size

Related reference

Estimators

Inferences about equality of proportions

Inferences about the proportions in populations are made using random samples of data drawn from each population of interest.

- Equality of proportions hypothesis test
A hypothesis test formally tests if the proportions in two or more populations are equal.

- Exact and asymptotic p-values
Asymptotic p-values are useful for large sample sizes when the calculation of an exact p-value is too computer-intensive.

- Wald, Score, Likelihood ratio
Three different ways are used to make large sample inferences all using a X^2 statistic, so it is important to understand the differences to avoid confusion.

Equality of proportions hypothesis test

A hypothesis test formally tests if the proportions in two or more populations are equal.

When one variable is an explanatory variable (X, fixed) and the other a response variable (Y, random), the hypothesis of interest is whether the populations have the same or different proportions in each category.

The hypotheses to test depends on the number of samples:

- For two samples, the null hypothesis states that the parameter of interest is equal to the hypothesized value, against the alternative hypothesis it is not equal to (or less than, or greater than) the hypothesized value.
You can formulate the hypotheses in terms of the parameter of interest: odds ratio = 1, the ratio of proportions = 1, or the difference of proportions = 0 depending on the desired effect size estimate.

- For more than two samples, the null hypothesis states that the proportions in each category are equal for all populations, against the alternative hypothesis that the proportions in a category are not equal for at least 2 populations.

When the test p-value is small, you can reject the null hypothesis and conclude that the populations differ in the proportions in at least one category.

Tests for contingency tables larger than 2 x 2 are omnibus tests and do not tell you which groups differ from each other or in which categories. You should use the mosaic plot to examine the association, or partition the contingency table into a series of 2 x 2 sub-tables and test each table.

Related concepts

Wald, Score, Likelihood ratio

Hypothesis testing

Related tasks

Testing equality of proportions (independent samples)

Testing equality of proportions (related samples)

Related reference

Tests for equality of proportions (independent samples)

Tests for equality of proportions (related samples)

Exact and asymptotic p-values

Asymptotic p-values are useful for large sample sizes when the calculation of an exact p-value is too computer-intensive.

Many test statistics follow a discrete probability distribution. However, hand calculation of the true probability distributions of many test statistics is too tedious except for small samples. It can be computationally difficult and time intensive even for a powerful computer. Instead, many statistical tests use an approximation to the true distribution. Approximations assume the sample size is large enough so that the test statistic converges to an appropriate limiting normal or chi-square distribution.

A p-value that is calculated using an approximation to the true distribution is called an asymptotic p-value. A p-value calculated using the true distribution is called an exact p-value. For large sample sizes, the exact and asymptotic p-values are very similar. For small sample sizes or sparse data, the exact and asymptotic p-values can be quite different and can lead to different conclusions about the hypothesis of interest.

When using the true distribution, due to the discreteness of the distribution, the p-value and confidence intervals are conservative. A conservative interval guarantees that the actual coverage level is at least as large as the nominal confidence level, though it can be much larger. For a hypothesis test, it guarantees protection from type I error at the nominal significance level. Some statisticians prefer asymptotic p-values and intervals, as they tend to achieve an average coverage closer to the nominal level. The problem is the actual coverage of a specific test or interval may be less than the nominal level.

Although a statistical test might commonly use an approximation, it does not mean it cannot be calculated using the true probability distribution. For example, an asymptotic p-value for the Pearson X^2 test uses the chi-squared approximation, but the test could also compute an exact p-value using the true probability distribution. Similarly, although the Fisher test is often called the Fisher exact test because it computes an exact p-value using the hypergeometric probability distribution, the test could also compute an asymptotic p-value. Likewise, the Wilcoxon-Mann-Whitney test often computes an exact p-value for small sample sizes and reverts to an asymptotic p-value for large sample sizes.

Wald, Score, Likelihood ratio

Three different ways are used to make large sample inferences all using a X^2 statistic, so it is important to understand the differences to avoid confusion.

The score, likelihood ratio, and Wald evidence functions are useful when analyzing categorical data. All have approximately chi-squared (X^2) distribution when the sample size is sufficiently large.

Because of their underlying use of the X^2 distribution, mixing of the techniques often occurs when testing hypotheses and constructing interval estimates. Introductory textbooks often use a score evidence function for the hypothesis test then use a Wald evidence function for the confidence interval. When you use different evidence functions the results can be inconsistent. The hypothesis test may be statistically significant, but the confidence interval may include the hypothesized value suggesting the result is not significant. Where possible you should use the same underlying evidence function to form the confidence interval and test the hypotheses.

Note that when the degrees of freedom is 1 the Pearson X^2 statistic is equivalent to a squared score Z statistic because they are both based on the score evidence function. A 2-tailed score Z test produces the same p-value as a Pearson X^2 test.

Many Wald-based interval estimates have poor coverage and defy conventional wisdom. Modern computing power allows the use of computationally intensive score evidence functions for interval estimates, such that we recommended them for general use.

Tests for equality of proportions (independent samples)

Tests for the equality of proportions of independent samples and their properties and assumptions.

Test

Purpose

Fisher exact

Test if the two proportions are equal.
Assumes fixed marginal distributions. Although not naturally fixed in most studies, this test still applies by conditioning on the marginal totals.

Uses hypergeometric distribution and computes an exact p-value. The exact p-value is conservative, that is, the actual rejection rate is below the nominal significance level. Recommended for small sample sizes or sparse data.

Score Z

Test if the two proportions are equal.
Uses the score statistic and computes an asymptotic p-value.

Note that the two-sided Score Z test is equivalent to Pearson's X^2 test.

Pearson X²

Test if the proportions are equal in each category.
Uses the score statistic and computes an asymptotic p-value.

Note that the asymptotic distribution of the X^2 test is the same regardless of whether only the total is fixed, both marginal distributions are fixed, or only one marginal distribution is fixed.

Related concepts

Equality of proportions hypothesis test

Exact and asymptotic p-values

Continuity correction

Related tasks

Testing equality of proportions (independent samples)

Testing equality of proportions (independent samples)

Test if there is a difference between the proportions of two or more independent samples.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups, and then click the hypothesis test.
The analysis task pane opens.

3. In the Y drop-down list, select the categorical response variable.

4. In the X drop-down list, select the categorical factor variable that identifies the groups.

5. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

6. In the Hypotheses drop-down list, select the null and alternative hypothesis.

7. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

8. Click Calculate.

Related concepts

Equality of proportions hypothesis test

Related tasks

Testing equality of proportions (related samples)

Related reference

Dataset layout

Tests for equality of proportions (independent samples)

Tests for equality of proportions (related samples)

Tests for the equality of proportions of related samples and their properties and assumptions.

Test

Purpose

McNemar-Mosteller Exact

Test if two proportions are equal.
Uses a binomial distribution and computes an exact p-value. The exact p-value is conservative, that is, the actual rejection rate is below the nominal significance level.

Recommended for small sample sizes or sparse data.

Score Z

Test if two proportions are equal.
Uses the score statistic and computes an asymptotic p-value.

Note that the two-sided Score Z test is equivalent to McNemar's X^2 test.

Related concepts

Equality of proportions hypothesis test

Exact and asymptotic p-values

Continuity correction

Related tasks

Testing equality of proportions (related samples)

Testing equality of proportions (related samples)

Test if there is a difference between the proportions of two or more related samples.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Pairs, and then click the hypothesis test.
The analysis task pane opens.

3. If the data consists of paired/repeated measurements in separate variables:

1. In the Model drop-down menu, select Paired / Repeated.

2. In the Y lists, select two categorical variables.

4. If the data consist of matched pairs with a separate response variable, factor variable, and blocking variable:

1. In the Model drop-down menu, select Matched.

2. In the Y drop-down list, select the categorical response variable.

3. In the X (factor) drop-down list, select the categorical factor variable identifying the groups.

4. In the Block drop-down list, select the blocking variable identifying the matching.

5. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

6. In the Hypotheses drop-down list, select the null and alternative hypothesis.

7. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

8. Click Calculate.

Related concepts

Equality of proportions hypothesis test

Related tasks

Testing equality of proportions (independent samples)

Related reference

Dataset layout

Tests for equality of proportions (related samples)

Inferences about independence

Inferences about the independence between categorical variables are made using a random bivariate sample of data drawn from the population of interest.

- Independence hypothesis test
A hypothesis test formally tests whether 2 categorical variables are independent.

- Continuity correction
Continuity corrections such as Yates X^2are no longer needed with modern computing power.

Independence hypothesis test

A hypothesis test formally tests whether 2 categorical variables are independent.

When both variables are random, the hypothesis of interest is whether the variables are independent.

The null hypothesis states that the variables are independent, against the alternative hypothesis that the variables are not independent. That is, in terms of the contingency table, the null hypothesis states the event "an observation is in row i" is independent of the event "that same observation is in column j," for all i and j, against the alternative hypothesis that is the negation of the null hypothesis.

When the test p-value is small, you can reject the null hypothesis and conclude that the variables are not independent.

Related concepts

Hypothesis testing

Related tasks

Testing independence

Related reference

Tests for independence

Continuity correction

Continuity corrections such as Yates X^2are no longer needed with modern computing power.

Continuity corrections have historically been used to make adjustments to the p-value when a continuous distribution approximates a discrete distribution. Yates correction for the Pearson chi-square (X^2) test is probably the most well-known continuity correction. In some cases, the continuity correction may adjust the p-value too far, and the test then becomes overly conservative.

Modern computing power makes such corrections unnecessary, as exact tests that use the discrete distributions are available for moderate and in many cases even large sample sizes. Hirji (Hirji 2005), states "An applied statistician today, in our view, may regard such corrections as interesting historical curiosities."

Related information

Hirji, K. F. (2005). Exact analysis of discrete data. CRC Press.

Tests for independence

Tests for the independence of 2 categorical variables and their properties and assumptions.

Test

Purpose

Pearson X²

Test if the variables are independent.
Uses the score statistic and computes an asymptotic p-value.

Likelihood ratio G²

Test if the variables are independent.
Uses the likelihood ratio statistic and computes an asymptotic p-value.   Pearson X² usually converges to the chi-squared distribution more quickly than G². The likelihood ratio test is often used in statistical modeling as the G² statistic is easier to compare between different models.

Related concepts

Independence hypothesis test

Hypothesis testing

Exact and asymptotic p-values

Continuity correction

Related tasks

Testing independence

Testing independence

Test if two categorical variables are independent in a bivariate population represented by the sample.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups, and then click the hypothesis test.
The analysis task pane opens.

3. In the X and Y drop-down lists, select the categorical variables.

4. In the Hypotheses drop-down list, select the independence null and alternative hypotheses (the 2nd in the list).

5. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

6. Click Calculate.

Related concepts

Independence hypothesis test

Related reference

Dataset layout

Tests for independence

Mosaic plot

A mosaic plot is a visual representation of the association between two variables.

A mosaic plot is a square subdivided into rectangular tiles the area of which represents the conditional relative frequency for a cell in the contingency table. Each tile is colored to show the deviation from the expected frequency (residual) from a Pearson X² or Likelihood Ratio G² test.

You can use the mosaic plot to discover the association between two variables. Red tiles indicate significant negative residuals, where the frequency is less than expected. Blue tiles indicate significant positive residuals, where the frequency is greater than expected. The intensity of the color represents the magnitude of the residual.

Related tasks

Creating a mosaic plot

Creating a mosaic plot

Visualize the association between 2 categorical variables.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Compare Groups, and then click Mosaic.
The analysis task pane opens.

3. In the Y drop-down list, select the categorical response variable.

4. In the X drop-down list, select the categorical factor variable identifying the groups.

5. In the Color by drop-down list, select Category to color by category, or Residual to color by the Pearson X^2 residual.

6. To label the tiles with the conditional relative frequency, click Label tiles.

7. Click Calculate.

Related concepts

Mosaic plot

Related reference

Dataset layout

Study design

Contingency analysis study requirements and dataset layout.

Requirements

- 2 categorical variables.

or:

- A qualitative response variable.

- A categorical factor variable with 2 or more levels.

or:

- 2 or more repeated measurements on a qualitative response variable.

Dataset layout

Use a column for each variable (Eye color, Hair color); each row has the values of the variables.

Subject (optional)

Eye color

Hair color

1

Brown

Black

2

Blue

Red

3

Blue

Blonde

4

Brown

Red

5

Green

Blonde

6

Hazel

Brown

7

Blue

Blonde

8

Brown

Black

9

Green

Red

10

Green

Blonde

…

…

…

Frequency form dataset layout

Use a column for each variable (Eye color, Hair color) and a column for the number of cases (Frequency); each row has the values of the variables and the frequency count.

Eye color

Hair color

Frequency

Brown

Black

69

Brown

Brown

119

Brown

Red

26

Brown

Blonde

7

Blue

Black

20

Blue

Brown

84

Blue

Red

17

Blue

Blonde

94

Hazel

Black

15

Hazel

Brown

54

Hazel

Red

14

Hazel

Blonde

10

Green

Black

5

Green

Brown

29

Green

Red

14

Green

Blonde

16

Chapter 13. Correlation and association

Correlation analysis explores the association between two or more variables and makes inferences about the strength of the relationship.

Note: It is common to use the terms correlation and association interchangeably. Technically, association refers to any relationship between two variables, whereas correlation is often used to refer only to a linear relationship between two variables. The terms are used interchangeably in this guide, as is common in most statistics texts.

- Scatter plot
A scatter plot shows the association between two variables. A scatter plot matrix shows all pairwise scatter plots for many variables.

- Covariance
Covariance is a measure of how much two variables change together. A covariance matrix measures the covariance between many pairs of variables.

- Correlation coefficient
A correlation coefficient measures the association between two variables. A correlation matrix measures the correlation between many pairs of variables.

- Inferences about association
Inferences about the strength of association between variables are made using a random bivariate sample of data drawn from the population of interest.

Scatter plot

A scatter plot shows the association between two variables. A scatter plot matrix shows all pairwise scatter plots for many variables.

If the variables tend to increase and decrease together, the association is positive. If one variable tends to increase as the other decreases, the association is negative. If there is no pattern, the association is zero.

When a straight line describes the relationship between the variables, the association is linear. When a constantly increasing or decreasing nonlinear function describes the relationship, the association is monotonic. Other relationships may be nonlinear or non-monotonic.

The type of relationship determines the statistical measures and tests of association that are appropriate.

If the association is a linear relationship, a bivariate normal density ellipse summarizes the correlation between variables. The narrower the ellipse, the greater the correlation between the variables. The wider and more round it is, the more the variables are uncorrelated. If the association is nonlinear, it is often worth trying to transform the data to make the relationship linear as there are more statistics for analyzing linear relationships and their interpretation is easier than nonlinear relationships.

An observation that appears detached from the bulk of observations may be an outlier requiring further investigation. An individual observation on each of the variables may be perfectly reasonable on its own but appear as an outlier when plotted on a scatter plot. Outliers can badly affect the product-moment correlation coefficient, whereas other correlation coefficients are more robust to them.

Related concepts

Correlation coefficient

Related tasks

Creating a scatter plot or scatter plot matrix

Creating a scatter plot or scatter plot matrix

Visualize the relationships between two or more variables.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Correlation, and then click Scatterplot or Scatterplot Matrix, and then click the plot type.
The analysis task pane opens.

3. If the model is bivariate, in the X and Y drop-down lists, select the variables.

4. If the model is multivariate, in the Variables list, select the variables.

5. Optional: To show stratification of the observations, select Vary point color/symbol, and then in the Group / Color / Symbol drop-down list, select a variable.

6. Optional: To label the observations, select the Label observations check box.

7. Click Calculate.

Related concepts

Scatter plot

Related reference

Dataset layout

Covariance

Covariance is a measure of how much two variables change together. A covariance matrix measures the covariance between many pairs of variables.

When the variables tend to show similar behavior, the covariance is positive. That is when greater values of one variable mainly correspond with the greater values of the other variable, or lesser values of one variable correspond with lesser values of the other variable. When the variables tend to show opposite behavior, the covariance is negative. That is when the greater values of one variable mainly correspond to the lesser values of the other and vice-versa.

The magnitude of the covariance is not meaningful to interpret. However, the standardized version of the covariance, the correlation coefficient, indicates by its magnitude the strength of the relationship.

A covariance matrix measures the covariance between many variables. Because the covariance of a variable with itself is that variable's variance, the diagonal of the covariance matrix is simply the variance of each variable.

Related concepts

Correlation coefficient

Related tasks

Calculating a covariance matrix

Correlation coefficient

A correlation coefficient measures the association between two variables. A correlation matrix measures the correlation between many pairs of variables.

The type of relationship between the variables determines the best measure of association:

- When the association between the variables is linear, the product-moment correlation coefficient describes the strength of the linear relationship.
The correlation coefficient ranges from -1 to +1. +1 indicates a perfect positive linear relationship, and -1 indicates a perfect negative linear relationship. Zero indicates the variables are uncorrelated and there is no linear relationship. Normally the correlation coefficient lies somewhere between these values.

- When the association between the variables is not linear, a rank correlation coefficient describes the strength of association.
Rank correlation coefficients range from -1 to +1. A positive rank correlation coefficient describes the extent to which as one variable increases the other variable also tends to increase, without requiring that increase to be linear. If one variable increases, as the other tends to decrease, the rank correlation coefficient is negative.

It is best to use a scatter plot to identify the type of association between the variables and then use an appropriate measure of association for the relationship. Do not be tempted just to look for the highest correlation coefficient.

A correlation matrix measures the correlation between many variables. It is equivalent to a covariance matrix of the standardized variables.

Related concepts

Scatter plot

Related tasks

Calculating a correlation matrix

Color map

A color map highlights the sign and magnitude of coefficients in a patterned matrix.

Red indicate negative values, blue indicates positive values. Intensity of the color represents the magnitude of the value, the darker more extreme.

Calculating a correlation matrix

Measure the correlation / association between 2 or more variables.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Correlation or Multivariate, and then click the statistics to show:

Option

Description

Correlation Matrix

Show the product-moment correlation coefficients.

Rank Correlation Matrix

Show the rank correlation coefficients.

The analysis task pane opens.

3. In the Variables list, select the variables.

4. Optional: To highlight the coefficients depending on their sign and magnitude, select the Color maps check box.

5. Optional: To show univariate descriptive statistics for each variable, select the Descriptives for each variable check box.

6. Click Calculate.

Related concepts

Correlation coefficient

Color map

Related reference

Dataset layout

Calculating a covariance matrix

Measure the covariance between 2 or more variables.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Multivariate, and then click Covariance Matrix.
The analysis task pane opens.

3. In the Variables list, select the variables.

4. Click Calculate.

Related concepts

Covariance

Related reference

Dataset layout

Inferences about association

Inferences about the strength of association between variables are made using a random bivariate sample of data drawn from the population of interest.

- Correlation not Causation
Do not be fooled into thinking a significant correlation implies you to make statements about causation.

- Parameter estimate
A parameter estimate is either a point or interval estimate of the unknown population correlation coefficient.

- Correlation/association hypothesis test
A hypothesis test formally tests if there is correlation/association between two variables in a population.

Correlation not Causation

Do not be fooled into thinking a significant correlation implies you to make statements about causation.

Correlation does not imply causation is a reminder that although a statistically significant correlation might exist between two variables, it does not imply that one causes the other.

There are many reasons why correlation does not imply causation: reverse causation where causation is actually in the opposite direction; bidirectional causation where a change in one variable causes a change in the other and vice-versa; a third unobserved variable that is the actual cause of the correlation; or simply a coincidence.

Correlation is a hint to investigate further, where possible using a controlled experiment. In some fields controlled experiments are impossible. Instead, cause and effect can be established only by observing the same thing in a wide variety of scenarios until all but the suspected cause can be ruled out. Even then the causal relationship can only be confirmed when you discover the underlying mechanism of the cause and effect.

Related concepts

Correlation and association

Parameter estimate

A parameter estimate is either a point or interval estimate of the unknown population correlation coefficient.

A point estimate is a single value that is the best estimate of the true unknown population correlation coefficient; a confidence interval is a range of values and indicates the uncertainty of the estimate.

Related concepts

Point and interval estimation

Related tasks

Estimating the correlation/association

Related reference

Estimators for correlation and association

Estimators for correlation and association

Estimators for correlation/association and their properties and assumptions.

Estimator

Purpose

Pearson r

Estimate the Pearson rho correlation coefficient using the sample Pearson correlation coefficient r.
Use when a linear function best describes the relationship between the variables.

Susceptible to outliers. Assumes a bivariate normal distribution.

Spearman rs

Estimate the Spearman rho rank correlation coefficient using the sample Spearman rs correlation coefficient.
Use when a monotonic function best describes the relationship between variables.

Kendall tau

Estimate the Kendall tau rank correlation coefficient.
Use when a monotonic function best describes the relationship between the variables. Preferred instead of Spearman correlation because Kendall's tau is the difference between the probability that the two variables are in the same order, versus the probability that the two variables are in different orders.

Robust to outliers.

Related concepts

Parameter estimate

Related tasks

Estimating the correlation/association

Estimating the correlation/association

Estimate the correlation/association between pairs of variables.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Correlation, and then the parameter estimator.
The analysis task pane opens.

3. If analyzing two variables:

1. In the Model drop-down menu, select Bivariate.

2. In the X drop-down list, select the first variable.

3. In the Y drop-down list, select the second variable.

4. If analyzing more than two variables:

1. In the Model drop-down menu, select Multivariate.

2. In the Variables list, select the variables.

5. In the Confidence interval edit box, type the confidence level as a percentage, or type - to suppress the confidence interval, and then in the drop-down list, select the confidence bounds.

6. In the Method drop-down list, select the interval estimator.

7. Click Calculate.

Related concepts

Parameter estimate

Related reference

Dataset layout

Estimators for correlation and association

Correlation/association hypothesis test

A hypothesis test formally tests if there is correlation/association between two variables in a population.

The hypotheses to test depends on the type of association:

- For a product-moment correlation, the null hypothesis states that the population correlation coefficient is equal to a hypothesized value (usually 0 indicating no linear correlation), against the alternative hypothesis that it is not equal (or less than, or greater than) the hypothesized value.

- For rank correlation methods, the hypotheses are restricted and more general. The null hypothesis states the variables are independent, against the alternative hypothesis that there is an association, such as a monotonic function.

When the test p-value is small, you can reject the null hypothesis and conclude that the population correlation coefficient is not equal to the hypothesized value, or for rank correlation that the variables are not independent. It is important to remember that a statistically significant test may not have any practical importance if the correlation coefficient is very small.

Pearson's and Kendall's tests are preferred as both have associated estimators of the population correlation coefficient (rho and tau respectively). Although the Spearman test is popular due to the ease of computation, the Spearman correlation coefficient is a measure of the linear association between the ranks of the variables and not the measure of association linked with the Spearman test.

Related concepts

Hypothesis testing

Related tasks

Testing correlation/association

Testing correlation/association

Test if the correlation between 2 variables is equal to a hypothesized value, or test for independence.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Correlation, and then the parameter estimator.
The analysis task pane opens.

3. If analyzing two variables:

1. In the Model drop-down menu, select Bivariate.

2. In the X drop-down list, select the first variable.

3. In the Y drop-down list, select the second variable.

4. If analyzing more than two variables:

1. In the Model drop-down menu, select Multivariate.

2. In the Variables list, select the variables.

5. Select the Hypothesis test check box.

6. In the Hypotheses drop-down list, select the null and alternative hypothesis.

7. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

8. Click Calculate.

Related concepts

Correlation/association hypothesis test

Related reference

Dataset layout

Study design

Multivariate analysis study requirements and dataset layout.

Requirements

- 2 or more quantitative variables.

Dataset layout

Use a a column for each variable (Height, Weight); each row has the values of the variables for a case (Subject).

Subject (optional)

Height

Weight

1

175

65

2

180

70

3

160

90

4

190

55

5

180

100

6

150

55

7

140

75

8

160

80

9

165

80

10

180

95

…

…

…

Chapter 14. Principal component analysis (PCA)

Principal component analysis (PCA) reduces the dimensionality of a dataset with a large number of interrelated variables while retaining as much of the variation in the dataset as possible.

PCA is a mathematical technique that reduces dimensionality by creating a new set of variables called principal components. The first principal component is a linear combination of the original variables and explains as much variation as possible in the original data. Each subsequent component explains as much of the remaining variation as possible under the condition that it is uncorrelated with the previous components.

The first few principal components provide a simpler picture of the data than trying to understand all the original variables. Sometimes, it is desirable to try and name and interpret the principal components, a process call reification, although this should not be confused with the purpose of factor analysis.

- Principal components
Principal components are the linear combinations of the original variables.

- Scree plot
A scree plot visualizes the dimensionality of the data.

- Biplot
A biplot simultaneously plots information on the observations and the variables in a multidimensional dataset.

- Monoplot
A monoplot plots information on the observations or the variables in a multidimensional dataset.

Principal components

Principal components are the linear combinations of the original variables.

Variances

Variances of each principal component show how much of the original variation in the dataset is explained by the principal component.

When the data is standardized, a component with a variance of 1 indicates that the principal component accounts for the variation equivalent to one of the original variables. Also, the sum of all the variances is equal to the number of original variables.

Coefficients

Coefficients are the linear combinations of the original variables that make up the principal component. The coefficients for each principal component can sometimes reveal the structure of the data. Absolute values near zero indicate that a variable contributes little to the component, whereas larger absolute values indicate variables that contribute more to the component.

Often, when the data is centered and standardized, the coefficients are normalized so that the sum of the squares of the coefficients of a component is equal to the variance of the component. In this normalization, the coefficients can be interpreted as the correlation between the original variable and the principal component, and are often called loadings (a term borrowed from factor analysis).

Scores

Scores are new variables that are the value of the linear combination of the original variables. The scores are normalized so that the sum of squares equals the variance of the principal component.

Related concepts

Scree plot

Related tasks

Calculating principal components

Related information

Jolliffe, I. (2002). Principal component analysis. Springer-Verlag, New York, Inc.

Scree plot

A scree plot visualizes the dimensionality of the data.

The scree plot shows the cumulative variance explained by each principal component. You can make decision on the number of components to keep to adequately describe a dataset using ad-hoc rules such as components with a variance > 0.7 or where the cumulative proportion of variation is > 80% or > 90% (Jolliffe 2002)

Related concepts

Principal components

Related tasks

Calculating principal components

Related information

Jolliffe, I. (2002). Principal component analysis. Springer-Verlag, New York, Inc.

Calculating principal components

Reduce the dimensionality of multidimensional data using PCA.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Multivariate, and then click Principal Components.
The analysis task pane opens.

3. In the Variables list, select the variables.

4. Optional: To highlight the coefficients depending on their sign and magnitude, select the Color maps check box.

5. Optional: To show a plot of the cumulative variance explained by each principal component, select the Scree plot check box.

6. Click Calculate.

7. To save the computed principal component scores back to the dataset for further analysis, on the Analyse-it ribbon tab, click Save Variable > Scores and select the number of dimensions to store.

Related concepts

Principal components

Scree plot

Color map

Related reference

Dataset layout

Biplot

A biplot simultaneously plots information on the observations and the variables in a multidimensional dataset.

A biplot can optimally represent any two of the following characteristics:

- distances between observations

- relationships between variables

- inner products between observations and variables

There are 3 types of biplot based on which of these characteristics they represent:

Type

Characteristics

PCA

Distances between the observations and also the inner products between observations and variables.

Covariance / Correlation

Relationships between the variables and the inner products between observations and variables.

Joint

Distances between observations and also the relationship between variables.

A 2-dimensional biplot represents the information contained in two of the principal components. It is an approximation of the original multidimensional space.

Classic biplot

The classical biplot (Gabriel 1971) plots points representing the observations and vectors representing the variables.

PCA biplot

A more recent innovation, the PCA biplot (Gower & Hand 1996), represents the variables with calibrated axes and observations as points allowing you to project the observations onto the axes to make an approximation of the original values of the variables.

Related concepts

Monoplot

Related tasks

Creating a biplot

Related information

Gower, J. C., Lubbe, S. G., & Le Roux, N. J. (2011). Understanding Biplots. John Wiley & Sons.

Gabriel, K. R. (1971). The biplot graphic display of matrices with application to principal component analysis. Biometrika, 58(3), 453-467.

Monoplot

A monoplot plots information on the observations or the variables in a multidimensional dataset.

A monoplot can only represent either of the following characteristics - distance between observations, or relationships between variables. In contrast, a joint biplot can represent both characteristics.

A 2-dimensional monoplot represents the information contained in two of the principal components. It is an approximation of the original multidimensional space.

The correlation monoplot plots vectors pointing away from the origin to represent the original variables. The angle between the vectors is an approximation of the correlation between the variables. A small angle indicates the variables are positively correlated, an angle of 90 degrees indicates the variables are not correlated, and an angle close to 180 degrees indicates the variables are negatively correlated. The length of the line and its closeness to the circle indicate how well the plot represents the variable. It is, therefore, unwise to make inferences about relationships involving variables with poor representation.

The covariance monoplot plots vectors pointing away from the origin to represent the original variables. The length of the line represents the variance of the variable, and the inner product between the vectors represents the covariance.

Related concepts

Biplot

Related tasks

Creating a correlation monoplot

Related information

Gower, J. C., Lubbe, S. G., & Le Roux, N. J. (2011). Understanding Biplots. John Wiley & Sons.

Creating a biplot

A biplot simultaneously shows information on the observations and the variables in a multidimensional dataset.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Multivariate > Biplot / Monoplot, and then click the plot type.
The analysis task pane opens.

3. In the Variables list, select the variables.

4. Optional: To label the observations, select the Label points check box.

5. Optional: To label the variables, select the Label vectors check box.

6. Optional: To show stratification, select Vary point color/symbol, and then in the Group / Color / Symbol drop-down list, select a variable.

7. Click Calculate.

Related concepts

Biplot

Related reference

Dataset layout

Creating a correlation monoplot

A correlation monoplot shows the relationship between many variables in a multidimensional dataset.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Multivariate > Biplot / Monoplot, and then click Correlation Monoplot.
The analysis task pane opens.

3. In the Variables list, select the variables.

4. Click Calculate.

Related concepts

Monoplot

Related reference

Dataset layout

Chapter 15. Factor analysis (FA)

Factor analysis (FA) explores the underlying factors (latent variables) among a large number of interrelated variables.

Factor analysis is related to principal component analysis (PCA), but the two are not identical and have different purposes. FA makes the assumption that an underlying model exists, whereas PCA is a descriptive variable reduction technique.

- EFA
Exploratory factor analysis (EFA) identifies the underlying relationships between a large number of interrelated variables when there are no prior hypotheses about factors or patterns amongst the variables.

- Factor rotation
Rotations minimize the complexity of the factor loadings to make the structure simpler to interpret.

EFA

Exploratory factor analysis (EFA) identifies the underlying relationships between a large number of interrelated variables when there are no prior hypotheses about factors or patterns amongst the variables.

EFA is a technique based on the common factor model which describes the measured variables by a function of the common factors, unique factors, and error of measurements. Common factors are those that influence two or more measured variables, while unique factors influence only one measured variable.

Pattern matrix

The factor pattern matrix loadings are the linear combinations of the factors that make up the original standardized variables.

Structure matrix

The factor structure matrix loadings are the correlation coefficients between the factors and the variables.

Correlation matrix

The factor correlation matrix coefficients are the correlation coefficients between the factors.

Note: When the factors are not rotated, or the rotation is orthogonal, there is no correlation between the factors and the correlation matrix is equal to the identity matrix. Also, the loadings in the pattern matrix and structure matrix are identical, although it can be useful to remember the different interpretations - as linear coefficients or correlation coefficients.

Related concepts

Factor rotation

Related tasks

Extracting factors

Factor rotation

Rotations minimize the complexity of the factor loadings to make the structure simpler to interpret.

Factor loading matrices are not unique, for any solution involving two or more factors there are an infinite number of orientations of the factors that explain the original data equally well. Rotation of the factor loading matrices attempts to give a solution with the best simple structure.

There are two types of rotation:

- Orthogonal rotations constrain the factors to be uncorrelated. Although often favored, in many cases it is unrealistic to expect the factors to be uncorrelated, and forcing them to be uncorrelated makes it less likely that the rotation produces a solution with a simple structure.

- Oblique rotations permit the factors to be correlated with one another. Often produces solutions with a simpler structure.

Related concepts

EFA

Related tasks

Extracting factors

Related reference

Matrix rotations

Matrix rotations

Orthogonal and oblique matrix rotations.

p = number of variables, m = number of factors.

Method

Parameters

Purpose

Varimax

Orthogonal only. A computational faster equivalent to CF-Varimax.

Promax

power 0 ... 4

Oblique only.

Crawford-Ferguson

kappa 0 ... 1

Smaller kappa minimizes variables complexity and larger kappa minimizes factor complexity.

CF-Varimax

Crawford-Ferguson kappa = 1/p.

Spread variances across factors. Each factor tends to have either large or small loadings on a particular variable making it easy to identify each variable with a single factor.

CF-Quartimax

Crawford-Ferguson kappa = 0.

Minimizes variable complexity. Works well with distinct clusters without cross-loadings.

CF-Equamax

Crawford-Ferguson kappa = m/(2p).

CF-Parsimax

Crawford-Ferguson kappa = (m-1) / (p+m-2).

CF-Factor Parsimony

Crawford-Ferguson kappa = 1.

Minimizes factor complexity. Primarily of theoretical interest.

Oblimin

gamma 0 ... 1

O-Quartimin

Oblimin gamma = 0.

Equivalent to quartimax.

O-Biquartimin

Oblimin gamma = 0.5.

O-Covarimin

Oblimin gamma = 1.

Equivalent to varimax.

Geomin

delta > 0

Minimizes variable complexity.

Related concepts

Factor rotation

Extracting factors

Extract the underlying factors (latent variables) among a large number of interrelated variables in a multidimensional dataset.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Multivariate, and then click Common Factors.
The analysis task pane opens.

3. In the Variables list, select the variables.

4. In the Factors to extract edit box, type the number of underlying factors to attempt to extract.

5. Optional: To highlight the coefficients depending on their sign and magnitude, select the Color maps check box.

6. Optional: To rotate the extracted factors to allow easier interpretation, in the Rotation drop-down list, select the type of rotation, and then in the Method drop-down list, select the rotation and type any additional required parameters.

7. Click Calculate.

Related concepts

EFA

Factor rotation

Color map

Related reference

Dataset layout

Chapter 16. Item reliability

Item reliability is the consistency of a set of items (variables); that is to what extent they measure the same thing. When a set of items are consistent, they can make a measurement scale such as a sum scale.

Cronbach’s alpha is the most popular measure of item reliability; it is the average correlation of items in a measurement scale. If the items have variances that significantly differ, standardized alpha is preferred.

When all items are consistent and measure the same thing, then the coefficient alpha is equal to 1. A high value for alpha does not imply that the measure is unidimensional. To prove that a scale is unidimensional, you can use factor analysis to check the dimensionality.

It is possible to see the effect of an individual item on the overall alpha value by recomputing Cronbach's alpha excluding that item. If alpha increases when you exclude an item, that item does not highly correlate with the other items in the scale. If the alpha decreases, that item does correlate with the other items in the scale.

Related tasks

Testing item reliability

Testing item reliability

Test the consistency of a set of items (variables).

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Multivariate, and then click Cronbach's Alpha.
The analysis task pane opens.

3. In the Variables list, select the variables.

4. If the variables have widely varying variances, select the Standardized check box.

5. To see the effect of each variable on the overall alpha, select the Effect of excluding each variable check box.

6. Click Calculate.

Related concepts

Item reliability

Related reference

Dataset layout

Chapter 17. Fit model

Fit model describes the relationship between a response variable and one or more predictor variables.

There are many different models that you can fit including simple linear regression, multiple linear regression, analysis of variance (ANOVA), analysis of covariance (ANCOVA), and binary logistic regression.

- Linear fit
A linear model describes the relationship between a continuous response variable and one or more explanatory variables using a linear function.

- Logistic / Probit fit
A model that describes the relationship between a categorical response variable and one or more explanatory variables using a logit or probit function.

Linear fit

A linear model describes the relationship between a continuous response variable and one or more explanatory variables using a linear function.

- Simple regression models
Simple regression models describe the relationship between a single predictor variable and a response variable.

- Advanced models
Advanced models describe the relationship between a response variable and multiple predictor terms.

- Scatter plot
A scatter plot shows the relationship between variables.

- Summary of fit
R² and similar statistics measure how much variability is explained by the model.

- Parameter estimates
Parameter estimates (also called coefficients) are the change in the response associated with a one-unit change of the predictor, all other predictors being held constant.

- Effect of model hypothesis test
An F-test formally tests the hypothesis of whether the model fits the data better than no model.

- Predicted against actual Y plot
A predicted against actual plot shows the effect of the model and compares it against the null model.

- Lack of Fit
An F-test or X^2-test formally tests how well the model fits the data.

- Effect of terms hypothesis test
An F-test formally tests whether a term contributes to the model.

- Effect leverage plot
An effect leverage plot, also known as added variable plot or partial regression leverage plot, shows the unique effect of a term in the model.

- Effect means
Effect means are least-squares estimates predicted by the model for each combination of levels in a categorical term, adjusted for the other model effects.

- Multiple comparisons
Multiple comparisons make simultaneous inferences about a set of parameters.

- Residual plot
A residual plot shows the difference between the observed response and the fitted response values.

- Residuals - normality
Normality is the assumption that the underlying residuals are normally distributed, or approximately so.

- Residuals - independence
Autocorrelation occurs when the residuals are not independent of each other. That is, when the value of e[i+1] is not independent from e[i].

- Outlier and influence plot
An influence plot shows the outlyingness, leverage, and influence of each case.

- Prediction
Prediction is the use of the model to predict the population mean or value of an individual future observation, at specific values of the predictors. Inverse prediction deals with the problem of predicting the value of a predictor for a given value of the response variable.

Simple regression models

Simple regression models describe the relationship between a single predictor variable and a response variable.

Name

Model

Purpose

Line

Y=b_0+b_1x

Fit a straight line.

Polynomial

Y=b_0+b_1x+b_1x2....

Fit a polynomial curve. Polynomials are useful when the function is smooth, a polynomial of a high enough degree estimates any smooth function. Polynomials are used as approximations and rarely represent a physical model.

Logarithmic

Y=b_0+ b_1Log(x)

Fit a logarithmic function curve.

Exponential

Y=a*b_1x

Fit an exponential function curve.

Power

Y=a*xb_1

Fit a power function curve.

Related tasks

Fitting a simple linear regression

Related reference

Advanced models

Fitting a simple linear regression

Fit a simple linear regression model to describe the relationship between single a single predictor variable and a response variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Fit Model, and then click the simple regression model.
The analysis task pane opens.

3. In the Y drop-down list, select the response variable.

4. In the X drop-down list, select the predictor variable.

5. Optional: To assign a weight to each row for the analysis, in the Weight drop-down list, select the weight variable.

6. Click Calculate.

Related concepts

Summary of fit

Parameter estimates

Effect of model hypothesis test

Residual plot

Related tasks

Plotting residuals

Identifying outliers and other influential points

Making predictions

Saving variables

Related reference

Dataset layout

Simple regression models

Advanced models

Advanced models describe the relationship between a response variable and multiple predictor terms.

An advanced model is built from simple terms, polynomial terms and their interactions.

Some common forms of advanced models are:

Type

Description

Simple regression

A single quantitative simple term (X).

Multiple regression

Two or more quantitative simple terms (X, Z)

Polynomial

A series of polynomial terms of the nth degree (X, X², X³...).

One-way ANOVA

A single categorical term (A).

Two-way ANOVA

Two categorical terms (A, B).

Two-way ANOVA with interaction

Two categorical terms and their interaction (A, B, A*B).

Fully factorial ANOVA

All the simple categorical terms and all the crossed interaction terms (A, B, C, A*B, A*C, B*C, A*B*C).

ANCOVA

One or more categorical terms and one or more quantitative terms (A, X) or (A, X, A*X).

Related tasks

Fitting a multiple linear regression

Performing ANOVA

Performing ANCOVA

Fitting an advanced linear model

Related reference

Simple regression models

Fitting a multiple linear regression

Fit a multiple linear regression model to describe the relationship between many quantitative predictor variables and a response variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Fit Model, and then click Multiple Regression.
The analysis task pane opens.

3. In the Y drop-down list, select the response variable.

4. In the Available variables list, select the predictor variables:

- To select a single variable, click the variable.

- To select multiple variables, click the first variable then hold down the CTRL key and click each additional variable.

- To select a range of variables, click the first variable then hold down the SHIFT key and click the last variable in the range.

5. Click Add Predictor.

6. Repeat steps 4 through 5 for each additional set of variables.

7. Optional: To assign a weight to each row for the analysis, in the Weight drop-down list, select the weight variable.

8. Click Calculate.

Related concepts

Summary of fit

Parameter estimates

Effect of model hypothesis test

Residual plot

Related tasks

Plotting residuals

Identifying outliers and other influential points

Making predictions

Saving variables

Related reference

Dataset layout

Advanced models

Performing ANOVA

Test if there is a difference between population means when a response variable is classified by one or more categorical variables (factors).

Note: Although the Fit Model analysis can be used to perform a simple 1-way ANOVA it is more intuitive to use Compare Groups > ANOVA analysis. Also, see the related tasks section for performing specific ANOVA models.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Fit Model, and then click ANOVA.
The analysis task pane opens.

3. In the Y drop-down list, select the response variable.

4. In the Available variables list, select the factor variable(s):

- To select a single variable, click the variable.

- To select multiple variables, click the first variable then hold down the CTRL key and click each additional variable.

- To select a range of variables, click the first variable then hold down the SHIFT key and click the last variable in the range.

5. Click Add Factor to add the simple factors.

6. To add the interaction between two or more factors, in the Terms list box, click the first term then hold down the CTRL key and click each additional term to include in the interaction, and then click Cross.

7. To remove a term, in the Terms list box, click the term and then click Remove.

Note: It is sometimes quicker to use the Factorial command to build a factorial model and then remove any unnecessary terms.

8. To remove a factor and all terms using the factor, in the Factor list box, click the factor and then click Remove.

When you select a factor, all terms that include the factor are selected. If you hold down the CTRL key and click additional factors, then all terms which include all the selected factors are selected.

9. Repeat steps 4 through 8 to build the model.

10. Click Calculate.

Related concepts

Effect of model hypothesis test

Effect of terms hypothesis test

Effect means

Related tasks

Performing 2-way or higher factorial ANOVA

Plotting main effects and interactions

Comparing effect means

Related reference

Dataset layout

Advanced models

Performing 2-way or higher factorial ANOVA

Test if there is a difference between population means when a response variable is classified by two or more categorical variables (factors).

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Fit Model, and then click ANOVA.
The analysis task pane opens.

3. In the Y drop-down list, select the response variable.

4. In the Available variables list, select the factor variables:

- To select a single variable, click the variable.

- To select multiple variables, click the first variable then hold down the CTRL key and click each additional variable.

- To select a range of variables, click the first variable then hold down the SHIFT key and click the last variable in the range.

5. Click Factorial, and then click Full for a full factorial model, or click the highest order of interaction terms to include in the model.

6. Optional: To sort the terms by order (that is, all simple terms, all two-way interactions, then all three-way interactions...) rather than in the order the variables were added, click Sort.

7. Click Calculate.

Related concepts

Effect of model hypothesis test

Effect of terms hypothesis test

Effect means

Related tasks

Performing ANOVA

Plotting main effects and interactions

Comparing effect means

Related reference

Dataset layout

Performing ANCOVA

Test if there is a difference between population means when a response variable is classified by one or more categorical variables (factors) while adjusting for the effect of one or more quantitative variables (covariates).

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Fit Model, and then click ANCOVA.
The analysis task pane opens.

3. In the Y drop-down list, select the response variable.

4. In the Available variables list, select the variable(s):

- To select a single variable, click the variable.

- To select multiple variables, click the first variable then hold down the CTRL key and click each additional variable.

- To select a range of variables, click the first variable then hold down the SHIFT key and click the last variable in the range.

5. To add the variables to the model as categorical factors, click Add Factor.

6. To add the variables to the model as quantitative covariates, click Add Covariate.

7. To add the interaction between two or more factors/covariates, in the Terms list box, click the first term then hold down the CTRL key and click each additional term to include in the interaction, and then click Cross.

8. To remove a term, in the Terms list box, click the term and then click Remove.

Note: It is sometimes quicker to use the Factorial command to build a model and then remove any unnecessary terms.

9. To remove a factor/covariate and all terms using the factor/covariate, in the Factors or Covariates list box, click the factor/covariate and then click Remove.

When you select a factor or covariate, all terms that include the factor or covariate are selected. If you hold down the CTRL key and click additional factors or covariates, then all terms which include all the selected factors or covariates are selected.

10. Repeat steps 4 through 9 to build the model.

11. Click Calculate.

Related concepts

Effect of model hypothesis test

Effect of terms hypothesis test

Effect means

Related tasks

Plotting main effects and interactions

Comparing effect means

Related reference

Dataset layout

Advanced models

Fitting an advanced linear model

Fit an advanced linear model to describe the relationship between many predictor variables and a response variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Fit Model, and then click Advanced.
The analysis task pane opens.

3. In the Y drop-down list, select the response variable.

4. In the Available variables list, select the predictor variable(s):

- To select a single variable, click the variable.

- To select multiple variables, click the first variable then hold down the CTRL key and click each additional variable.

- To select a range of variables, click the first variable then hold down the SHIFT key and click the last variable in the range.

5. Optional: Click the drop-down menu arrow next to the Available variables list, and then select the measurement scale of the variable.

If the measurement scale is not set, Analyse-it tries to determine if the variable is categorical or continuous scale. After you add a variable to a model, the icon alongside the variable indicates the assumed scale.

6. Click Add to add a variable, or click Factorial to add the factorial terms, or click Polynomial to add a polynomial terms.

7. To add the interaction between two or more variables, in the Terms list box, click the first term then hold down the CTRL key and click each additional term to include in the interaction, and then click Cross.

8. To remove a term, in the Terms list box, click the term and then click Remove.

9. Repeat steps 4 through 8 to build the model.

10. Optional: To assign a weight to each row for the analysis, in the Weight drop-down list, select the weight variable.

11. Click Calculate.

Related concepts

Summary of fit

Parameter estimates

Effect of model hypothesis test

Effect of terms hypothesis test

Related tasks

Plotting residuals

Identifying outliers and other influential points

Plotting main effects and interactions

Comparing effect means

Making predictions

Saving variables

Related reference

Dataset layout

Advanced models

Scatter plot

A scatter plot shows the relationship between variables.

The scatter plot identifies the relationship that best describes the data, whether a straight line, polynomial or some other function.

A scatter plot matrix shows the relationship between each predictor and the response, and the relationship between each pair of predictors. You can use the matrix to identify the relationship between variables, to identify where additional terms such as polynomials or interactions are needed, and to see if transformations are needed to make the predictors or response linear. The scatter plot matrix does not convey the joint relationship between each predictor and the response since it does not take into account the effect on the response of the other variables in the model. Effect leverage and residual plots fulfill this purpose after fitting the model.

Summary of fit

R² and similar statistics measure how much variability is explained by the model.

R² is the proportion of variability in the response explained by the model. It is 1 when the model fits the data perfectly, though it can only attain this value when all sets of predictors are different. Zero indicates the model fits no better than the mean of the response. You should not use R² when the model does not include a constant term, as the interpretation is undefined.

For models with more than a single term, R² can be deceptive as it increases as you add more parameters to the model, eventually reaching saturation at 1 when the number of parameters equals the number of observations. Adjusted R² is a modification of R² that adjusts for the number of parameters in the model. It only increases when the terms added to the model improve the fit more than would be expected by chance. It is preferred when building and comparing models with a different number of parameters.

For example, if you fit a straight-line model, and then add a quadratic term to the model, the value of R² increases. If you continued to add more the polynomial terms until there are as many parameters as the number of observations, then the R² value would be 1. The adjusted R² statistic is designed to take into account the number of parameters in the model and ensures that adding the new term has some useful purpose rather than simply due to the number of parameters approaching saturation.

In cases where each set of predictor values are not unique, it may be impossible for the R² statistic to reach 1. A statistic called the maximum attainable R² indicates the maximum value that R² can achieve even if the model fitted perfectly. It is related to the pure error discussed in the lack of fit test.

The root mean square error (RMSE) of the fit, is an estimate of the standard deviation of the true unknown random error (it is the square root of the residual mean square). If the model fitted is not the correct model, the estimate is larger than the true random error, as it includes the error due to lack of fit of the model as well as the random errors.

Parameter estimates

Parameter estimates (also called coefficients) are the change in the response associated with a one-unit change of the predictor, all other predictors being held constant.

The unknown model parameters are estimated using least-squares estimation.

A coefficient describes the size of the contribution of that predictor; a near-zero coefficient indicates that variable has little influence on the response. The sign of the coefficient indicates the direction of the relationship, although the sign can change if more terms are added to the model, so the interpretation is not particularly useful. A confidence interval expresses the uncertainty in the estimate, under the assumption of normally distributed errors. Due to the central limit theorem, violation of the normality assumption is not a problem if the sample size is moderate.

- For quantitative terms, the coefficient represents the rate of change in the response per 1 unit change of the predictor, assuming all other predictors are held constant. The units of measurement for the coefficient are the units of response per unit of the predictor.
For example, a coefficient for Height of 0.75, in a simple model for the response Weight (kg) with predictor Height (cm), could be expressed as 0.75 kg per cm which indicates a 0.75 kg weight increase per 1 cm in height.

When a predictor is a logarithm transformation of the original variable, the coefficient is the rate of change in the response per 1 unit change in the log of the predictor. Commonly base 2 log and base 10 log are used as transforms. For base 2 log the coefficient can be interpreted as the rate of change in the response when for a doubling of the predictor value. For base 10 log the coefficient can be interpreted as the rate of change in the response when the predictor is multiplied by 10, or as the % change in the response per % change in the predictor.

- For categorical terms, there is a coefficient for each level:

- For nominal predictors the coefficients represent the difference between the level mean and the grand mean.
Analyse-it uses effect coding for nominal terms (also known as the mean deviation coding). The sum of the parameter estimates for a categorical term using effect coding is equal to 0.

- For ordinal predictors, the coefficients represent the difference between the level mean and the baseline mean.
Analyse-it uses reference coding for ordinal terms. The first level is used as the baseline or reference level.

- For the constant term, the coefficient is the response when all predictors are 0, and the units of measurement are the same as the response variable.

A standardized parameter estimate (commonly known as standardized beta coefficient) removes the unit of measurement of predictor and response variables. They represent the change in standard deviations of the response for 1 standard deviation change of the predictor. You can use them to compare the relative effects of predictors measured on different scales.

VIF, the variance inflation factor, represents the increase in the variance of the parameter estimate due to correlation (collinearity) between predictors. Collinearity between the predictors can lead to unstable parameter estimates. As a rule of thumb, VIF should be close to the minimum value of 1, indicating no collinearity. When VIF is greater than 5, there is high collinearity between predictors.

A t-test formally tests the null hypothesis that the parameter is equal to 0, against the alternative hypothesis that it is not equal to 0. When the p-value is small, you can reject the null hypothesis and conclude that the parameter is not equal to 0 and it does contribute to the model.

When a parameter is not deemed to contribute statistically to the model, you can consider removing it. However, you should be cautious of removing terms that are known to contribute by some underlying mechanism, regardless of the statistical significance of a hypothesis test, and recognize that removing a term can alter the effect of other terms.

Related concepts

Point and interval estimation

Effect of model hypothesis test

An F-test formally tests the hypothesis of whether the model fits the data better than no model.

It is common to test whether the model fits the data better than a null model that just fits the mean of the response. An analysis of variance table partitions the total variance in the response variable into the variation after fitting the full model (called the model error, or residual), the variation after fitting the null model, and the reduction by fitting the full model compared to the null model.

An F-test formally tests whether the reduction is statistically significant. The null hypothesis states that all the parameters except the intercept are zero against the alternative that at least one parameter is not equal to zero. When the p-value is small, you can reject the null hypothesis and conclude that at least one parameter is not zero.

Related concepts

Predicted against actual Y plot

Hypothesis testing

ANOVA table

ANOVA table

An analysis of variance (ANOVA) table shows the sources of variation.

An ANOVA table uses various abbreviations:

Abbreviation

Description

SS

Sum of squares, the sum of the squared deviations from the expected value.

DF

Degrees of freedom, the number of values that are free to vary.

MS

Mean square, the sum of squares divided by the degrees of freedom.

F

F-statistic, the ratio of two mean squares that forms the basis of a hypothesis test.

p-value

p-value, the probability of obtaining an F statistic at least as extreme as that observed when the null hypothesis is true.

Predicted against actual Y plot

A predicted against actual plot shows the effect of the model and compares it against the null model.

For a good fit, the points should be close to the fitted line, with narrow confidence bands. Points on the left or right of the plot, furthest from the mean, have the most leverage and effectively try to pull the fitted line toward the point. Points that are vertically distant from the line represent possible outliers. Both types of points can adversely affect the fit.

The plot is also a visualization of the ANOVA table, except each observation is shown so you can gain much more insight than just the hypothesis test. The confidence interval around the full model line portrays the F-test that all the parameters except the intercept are zero. When the confidence interval does not include the horizontal null model line, the hypothesis test is significant.

Related concepts

Effect of model hypothesis test

Lack of Fit

An F-test or X^2-test formally tests how well the model fits the data.

When the model fitted is correct the residual (model error) mean square provides and unbiased estimate of the true variance. If the model is wrong, then the mean square is larger than the true variance. It is possible to test for lack of fit by comparing the model error mean square to the true variance.

When the true variance is known, a X^2-squared test formally tests whether the model error is equal to the hypothesized value.

When the true variance is unknown, and there are multiple observations for each set of predictor values, an F-test formally tests whether there is a difference between pure error and the model error. The pure error is the pooled variance calculated for all unique sets of predictor values. It makes no assumptions about the model but does assume that the variance is the same for each set of predictor values. An analysis of variance table shows the pure error, model error, and the difference between them called the lack of fit.

The null hypothesis states that the model error mean square is equal to the hypothesized value/pure error, against the alternative that it is greater than. When the test p-value is small, you can reject the null hypothesis and conclude that there is a lack of fit.

Related concepts

Hypothesis testing

ANOVA table

Effect of terms hypothesis test

An F-test formally tests whether a term contributes to the model.

In most modeling analyses the aim is a model that describes the relationship using as few terms as possible. It is therefore of interest to look at each term in the model to decide if the term is providing any useful information.

An analysis of variance table shows the reduction in error by including each term. An F-test for each term is a formal hypothesis test to determine if the term provides useful information to the model. The null hypothesis states that the term does not contribute to the model, against the alternative hypothesis that it does. When the p-value is small, you can reject the null hypothesis and conclude that the term does contribute to the model.

When a term is not deemed to contribute statistically to the model, you may consider removing it. However, you should be cautious of removing terms that are known to contribute by some underlying mechanism, regardless of the statistical significance of a hypothesis test, and recognize that removing a term can alter the effect of other terms.

You should choose the type of F-test based on what other terms you want the test to consider.

Test

Purpose

Partial

Measures the effect of the term adjusted for all other terms in the model. The sum of squares in the ANOVA table is known as the Type III sum of squares.
A partial F-test is equivalent to testing if all the parameter estimates for the term are equal to zero.

Sequential

Measures the effect of the term adjusting only for the previous terms included in the model. The sum of squares in the ANOVA table is known as the Type I sum of squares.
A sequential F-test is often useful when fitting a polynomial regression.

Note: The squared t-statistic for a coefficient t-test is equivalent to the F statistic when using the partial F-test. The t-test is not suitable when the model includes categorical variables coded as dummy predictor variables as each term consists multiple coefficient t-tests.

Related concepts

Effect leverage plot

Hypothesis testing

ANOVA table

Effect leverage plot

An effect leverage plot, also known as added variable plot or partial regression leverage plot, shows the unique effect of a term in the model.

A horizontal line shows the constrained model without the term; a slanted line shows the unconstrained model with the term. The plot shows the unique effect of adding a term to a model assuming the model contains all the other terms and the influence of each point on the effect of term hypothesis test. Points further from the horizontal line than the slanted line effectively try to make the hypothesis test more significant, and those closer to the horizontal than the slanted line try to make the hypothesis test less significant. When the confidence band fully encompasses the horizontal line, you can conclude the term does not contribute to the model. It is equivalent to a non-significant F-test for the partial effect of the term.

You can also use the plot to identify cases that are likely to influence the parameter estimates. Points furthest from the intersection of the horizontal and slanted lines have high leverage, and effectively try to pull the line towards them. A high leverage point that is distant from the bulk of the points can have a large influence the parameter estimates for the term.

Finally, you can use the plot to spot near collinearity between terms. When the terms are collinear, the points collapse toward a vertical line.

Related concepts

Effect of terms hypothesis test

Effect means

Effect means are least-squares estimates predicted by the model for each combination of levels in a categorical term, adjusted for the other model effects.

Least-squares means are predictions made at specific values of the predictors. For each categorical variable in the term, the value is the combination of levels. For all other variables that are not part of the term, the values are neutral values. The neutral value for a quantitative variable is the mean. The neutral value of a nominal variable is the average over all levels which is equivalent to values of 0 for each dummy predictor variable in an effect coding scheme. The neutral value of an ordinal variable is the reference level.

Related concepts

Multiple comparisons

Point and interval estimation

Related tasks

Plotting main effects and interactions

Comparing effect means

Plotting main effects and interactions

Plot the main effects and interactions to interpret the means.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Terms group, click Effect Means.
The analysis task pane Effect Means panel opens.

3. To show main effect plots for each factor, click the Main effect plots check box.

4. To show interaction effect plots for each 2-way interaction and up to a single 3-way interaction, click the Interaction effect plots check box.

5. To show the means plot for individual terms, in the Individual effects list box, select the individual term check boxes, and then select the Means plot check box.

6. To show the means table for individual terms, in the Individual effects list box, select the individual term check boxes, and then select the Means check box.

Note: For convenience, the Means and Means plot options apply to all selected Individual effects.

7. Click Recalculate.

Related concepts

Effect means

Multiple comparisons

Multiple comparisons make simultaneous inferences about a set of parameters.

When making inferences about more than one parameter (such as comparing many means, or the differences between many means), you must use multiple comparison procedures to make inferences about the parameters of interest. The problem when making multiple comparisons using individual tests such as Student's t-test applied to each comparison is the chance of a type I error increases with the number of comparisons. If you use a 5% significance level with a hypothesis test to decide if two groups are significantly different, there is a 5% probability of observing a significant difference that is simply due to chance (a type I error). If you made 20 such comparisons, the probability that one or more of the comparisons is statistically significant simply due to chance increases to 64%. With 50 comparisons, the chance increases to 92%. Another problem is the dependencies among the parameters of interest also alter the significance level. Therefore, you must use multiple comparison procedures to maintain the simultaneous probability close to the nominal significance level (typically 5%).

Multiple comparison procedures are classified by the strength of inference that can be made and the error rate controlled. A test of homogeneity controls the probability of falsely declaring any pair to be different when in fact all are the same. A stronger level of inference is confident inequalities and confident directions which control the probability of falsely declaring any pair to be different regardless of the values of the others. An even stronger level is a set of simultaneous confidence intervals that guarantees that the simultaneous coverage probability of the intervals is at least 100(1-alpha)% and also have the advantage of quantifying the possible effect size rather than producing just a p-value. A higher strength inference can be used to make an inference of a lesser strength but not vice-versa. Therefore, a confidence interval can be used to perform a confidence directions/inequalities inference, but a test of homogeneity cannot make a confidence direction/inequalities inference.

The most well known multiple comparison procedures, Bonferroni and Šidák, are not multiple comparison procedures per se. Rather they are an inequality useful in producing easy to compute multiple comparison methods of various types. In most scenarios, there are more powerful procedures available. A useful application of Bonferroni inequality is when there are a small number of pre-planned comparisons. In these cases, you can use the standard hypothesis test or confidence interval with the significance level (alpha) set to the Bonferroni inequality (alpha divided by the number of comparisons).

A side effect of maintaining the significance level is a lowering of the power of the test. Different procedures have been developed to maintain the power as high as possible depending on the strength of inference required and the number of comparisons to be made. All contrasts comparisons allow for any possible contrast; all pairs forms the k*(k-1)/2 pairwise contrasts, whereas with best forms k contrasts each with the best of the others, and against control forms k-1 contrasts each against the control group. You should choose the appropriate contrasts of interest before you perform the analysis, if you decide after inspecting the data, then you should only use all contrasts comparison procedures.

Related concepts

Effect means

Related tasks

Comparing effect means

Related reference

Multiple comparison procedures

Multiple comparison procedures

Methods of controlling the Type I error and dependencies between point estimates when making multiple comparisons between effect means in a linear model.

Procedure

Purpose / Assumptions

Student's t (Fisher's LSD)

Compare the means of each pair of groups using the Student's t method.
When making all pairwise comparisons this procedure is also known as unprotected Fisher's LSD, or when only performed following significant ANOVA F -test known as protected Fisher's LSD.

Controls the Type I error rate individually for each contrast.

Tukey-Kramer

Compare the means of all pairs of groups using the Tukey-Kramer method.
Controls the error rate simultaneously for all k(k+1)/2 contrasts.

Hsu

Compare the means of all groups against the best of the other groups using the Hsu method.
Controls the error rate simultaneously for all k contrasts.

Dunnett

Compare the means of all groups against a control using the Dunnett method.
Controls the error rate simultaneously for all k-1 contrasts.

Scheffé

Compare the means of all groups against all other groups using the Scheffé F method.
Controls the error rate simultaneously for all possible contrasts.

Bonferroni

Not a multiple comparisons method. It is an inequality useful in producing easy to compute multiple comparisons. In most scenarios, there are more powerful procedures such as Tukey, Dunnett, Hsu.
A useful application of Bonferroni inequality is when there are a small number of pre-planned comparisons. In this case, use the Student's t (LSD) method with the significance level (alpha) set to the Bonferroni inequality (alpha divided by the number of comparisons). In this scenario, it is usually less conservative than using Scheffé all contrast comparisons.

Related concepts

Multiple comparisons

Related tasks

Comparing effect means

Related information

Hsu, J. C. (1996). Multiple Comparisons: Theory and Methods. CRC Press.

Comparing effect means

Compare the effect means to make inferences about the differences between them.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Terms group, click Effect Means.
The analysis task pane Effect Means panel opens.

3. In the Individual effects list box, select the individual term check box, and then select the Multiple Comparisons check box.

4. In the Type drop-down list, select the comparisons, and then:

- If comparing with best, in the Best drop-down list, select the "best" group.

- If comparing against a control, in the Control drop-down list, select the control group.

5. Optional: To sort the contrasts by the magnitude of the difference, select the Ordered check box.

6. Optional: In the Method drop-down list, select the multiple comparisons procedure, and in the Confidence interval edit box, type the confidence level as a percentage, or type - to suppress the confidence interval, and then in the drop-down list, select the confidence bounds.

7. Optional: To perform a hypothesis test to test if the difference is equal to 0, select the Hypothesis test check box, and then in the Significance level edit box, type the predefined significance level against.

8. Optional: To show a mean-mean scatter plot, select the Mean-Mean scatter plot check box.

Note: For convenience the confidence interval, hypothesis test, and mean-mean scatter plot options apply to all selected Individual effects.

9. Click Calculate.

Related concepts

Multiple comparisons

Effect means

Related reference

Multiple comparison procedures

Residual plot

A residual plot shows the difference between the observed response and the fitted response values.

The ideal residual plot, called the null residual plot, shows a random scatter of points forming an approximately constant width band around the identity line.

It is important to check the fit of the model and assumptions – constant variance, normality, and independence of the errors, using the residual plot, along with normal, sequence, and lag plot.

Assumption

How to check

Model function is linear

The points form a pattern when the model function is incorrect.
You might be able to transform variables or add polynomial and interaction terms to remove the pattern.

Constant variance

If the points tend to form an increasing, decreasing or non-constant width band, then the variance is not constant.
You should consider transforming the response variable or incorporating weights into the model. When variance increases as a percentage of the response, you can use a log transform, although you should ensure it does not produce a poorly fitting model.

Even with non-constant variance, the parameter estimates remain unbiased if somewhat inefficient. However, the hypothesis tests and confidence intervals are inaccurate.

Normality

Examine the normal plot of the residuals to identify non-normality.
Violation of the normality assumption only becomes an issue with small sample sizes. For large sample sizes, the assumption is less important due to the central limit theorem, and the fact that the F- and t-tests used for hypothesis tests and forming confidence intervals are quite robust to modest departures from normality.

Independence

When the order of the cases in the dataset is the order in which they occurred:
Examine a sequence plot of the residuals against the order to identify any dependency between the residual and time.

Examine a lag-1 plot of each residual against the previous residual to identify a serial correlation, where observations are not independent, and there is a correlation between an observation and the previous observation.

Time-series analysis may be more suitable to model data where serial correlation is present.

For a model with many terms, it can be difficult to identify specific problems using the residual plot. A non-null residual plot indicates that there are problems with the model, but not necessarily what these are.

Related concepts

Residuals - normality

Residuals - independence

Outlier and influence plot

Related tasks

Plotting residuals

Residuals - normality

Normality is the assumption that the underlying residuals are normally distributed, or approximately so.

While a residual plot, or normal plot of the residuals can identify non-normality, you can formally test the hypothesis using the Shapiro-Wilk or similar test.

The null hypothesis states that the residuals are normally distributed, against the alternative hypothesis that they are not normally-distributed. If the test p-value is less than the predefined significance level, you can reject the null hypothesis and conclude the residuals are not from a normal distribution. If the p-value is greater than the predefined significance level, you cannot reject the null hypothesis.

Violation of the normality assumption only becomes an issue with small sample sizes. For large sample sizes, the assumption is less important due to the central limit theorem, and the fact that the F- and t-tests used for hypothesis tests and forming confidence intervals are quite robust to modest departures from normality.

Related concepts

Residual plot

Residuals - independence

Outlier and influence plot

Related tasks

Plotting residuals

Residuals - independence

Autocorrelation occurs when the residuals are not independent of each other. That is, when the value of e[i+1] is not independent from e[i].

While a residual plot, or lag-1 plot allows you to visually check for autocorrelation, you can formally test the hypothesis using the Durbin-Watson test. The Durbin-Watson statistic is used to detect the presence of autocorrelation at lag 1 (or higher) in the residuals from a regression. The value of the test statistic lies between 0 and 4, small values indicate successive residuals are positively correlated. If the Durbin-Watson statistic is much less than 2, there is evidence of positive autocorrelation, if much greater than 2 evidence of negative autocorrelation.

The null hypothesis states that the residuals are not autocorrelated, against the alternative hypothesis that they are. If the test p-value is less than the predefined significance level, you can reject the null hypothesis and conclude the residuals are correlated. If the p-value is greater than the predefined significance level, you cannot reject the null hypothesis.
Note: The p-value is computed using the bootstrap method and can take a long time to compute.

Related concepts

Residual plot

Residuals - normality

Outlier and influence plot

Related tasks

Plotting residuals

Plotting residuals

Plot the residuals to check the fit and assumptions of the model.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Diagnostics group, click Residuals, and then click:

Option

Description

Residual

Plot the residuals.

3-up

Plot the residuals and a histogram and normal plot.

5-up

Plot the residuals, a histogram and normal plot, a sequence plot, and a lag plot.

The analysis task pane, the Residuals panel opens.

3. In the Residuals drop-down list, select:

Option

Description

Raw

Plot the raw residual Y - Y fitted.

Standardized

Plot the standardized residual (Y - Y fitted) / SE.

4. Optional: To test the normality of the residuals, On the Analyse-it ribbon tab, in the Diagnostics group, click Test Normality.

5. Optional: To test for autocorrelation amongst the residuals, On the Analyse-it ribbon tab, in the Diagnostics group, click Test Autocorrelation.

6. Click Recalculate.

Related concepts

Residual plot

Residuals - normality

Residuals - independence

Outlier and influence plot

An influence plot shows the outlyingness, leverage, and influence of each case.

The plot shows the residual on the vertical axis, leverage on the horizontal axis, and the point size is the square root of Cook's D statistic, a measure of the influence of the point.

Outliers are cases that do not correspond to the model fitted to the bulk of the data. You can identify outliers as those cases with a large residual (usually greater than approximately +/- 2), though not all cases with a large residual are outliers and not all outliers are bad. Some of the most interesting cases may be outliers.

Leverage is the potential for a case to have an influence on the model. You can identify points with high leverage as those furthest to the right. A point with high leverage may not have much influence on the model if it fits the overall model without that case.

Influence combines the leverage and residual of a case to measure how the parameter estimates would change if that case were excluded. Points with a large residual and high leverage have the most influence. They can have an adverse effect on (perturb) the model if they are changed or excluded, making the model less robust. Sometimes a small group of influential points can have an unduly large impact on the fit of the model.

Related concepts

Residual plot

Residuals - normality

Residuals - independence

Related tasks

Identifying outliers and other influential points

Identifying outliers and other influential points

Plot measures to identify cases with large outliers, high leverage, or major influence on the fitted model.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Diagnostics group, click Outliers and Influence.
The analysis task pane Outliers, Leverage, Influence panel opens.

3. Optional: To highlight and label cases that are large outliers, have high leverage or major influence on the fitted model, select the Identify influential cases check box.

4. Optional: To individually plot the Studentized residual, leverage, and Cook's D influence measures by observation number, select the Index plots check box.

5. Click Recalculate.

Points highlighted in red are deemed to be potentially influential to the fitted model.

Related concepts

Outlier and influence plot

Prediction

Prediction is the use of the model to predict the population mean or value of an individual future observation, at specific values of the predictors. Inverse prediction deals with the problem of predicting the value of a predictor for a given value of the response variable.

When making predictions, it is important that the data used to fit the model is similar to future populations to which you want to apply the prediction. You should be careful of making predictions outside the range of the observed data. Assumptions met for the observed data may not be met outside the range. Non-constant variance can cause confidence intervals for the predicted values to become unrealistically narrow or so large as to be useless. Alternatively, a different fit function may better describe the unobserved data outside the range.

When making multiple predictions of the population mean at different sets of predictor values the confidence intervals can be simultaneous or individual. A simultaneous interval ensures you achieve the confidence level simultaneously for all predictions, whereas individual intervals only ensure confidence for the individual prediction. With individual inferences, the chance of at least one interval not including the true value increases with the number of predictions.

Related tasks

Making predictions

Making inverse predictions

Making predictions

Predict the value of the mean at specific values of the predictors, or the value of an individual future observation.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, click Predict, and then click Y.
The analysis task pane Predict Y panel opens.

3. In the Terms list box, under the #1 column, type the value for each predictor.
You can leave a value blank in which case the average value is used. Leaving the value blank is useful in ANOVA models where you can predict the mean at specific combinations averaged over other effects.

4. To predict the expected mean, select the Mean check box, and in the Confidence interval edit box, type the confidence level. Finally, in the Method drop-down list, select Simultaneous for intervals that ensure you achieve the confidence level simultaneously for all predictions, or Individual for intervals only that ensure confidence for each individual prediction.

5. To predict the future value of an individual observation, select the Individual check box, and in the Prediction interval edit box, type the confidence level.

6. Optional: On the Fit panel, in the Predicted values drop-down list, select the style to plot the predicated values on the scatter plot.

7. Click Recalculate.

Related concepts

Prediction

Point and interval estimation

Making inverse predictions

Predict the values of the predictors at a specific value or values of the response variable.

You can only make inverse predictions when fitting a simple regression model.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, click Predict, and then click X given Y.
The analysis task pane Inverse Predict X given Y panel opens.

3. In the Predictions list box, under the Y column, type the values for the response variable.

4. Optional: In the Confidence interval edit box, type the confidence level, and then in the drop-down list, select the confidence bounds. Finally, in the Method drop-down list, select Simultaneous for intervals that ensure you achieve the confidence level simultaneously for all predictions, or Individual for intervals only that ensure confidence for each individual prediction.

5. Optional: On the Fit panel, in the Predicted values drop-down list, select the style to plot the predicated values on the scatter plot.

6. Click Recalculate.

Related concepts

Prediction

Saving variables

Save the values computed by the analysis back to the dataset for further analysis.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, click Save Variable, and then select the variable to store to the dataset.

Note: If the Save Variable command is shaded, On the Analyse-it ribbon tab, in the Report group, click Recalculate to update the analysis.

Logistic / Probit fit

A model that describes the relationship between a categorical response variable and one or more explanatory variables using a logit or probit function.

- Generalized linear models
A generalized linear model (GLM) is a generalization of the linear models by allowing the linear model to be related to the response variable via a link function and an error distribution other than a normal distribution. The unknown model parameters are estimated using maximum-likelihood estimation.

- Parameter estimates
Parameter estimates (also called coefficients) are associated with a one-unit change of the predictor, all other predictors being held constant.

- Odds ratio estimates
Odds ratios are the increase or decrease in odds associated with a change of the predictor, all other predictors been held constant.

- Effect of model hypothesis test
A likelihood ratio or Wald X² test formally tests the hypothesis of whether the model fits the data better than no model.

- Effect of term hypothesis test
A likelihood ratio or Wald X² test formally tests the hypothesis of whether a term contributes to the model.

Generalized linear models

A generalized linear model (GLM) is a generalization of the linear models by allowing the linear model to be related to the response variable via a link function and an error distribution other than a normal distribution. The unknown model parameters are estimated using maximum-likelihood estimation.

Type

Description

Logit / Logistic

Fit a model to a binary response variable expressed by the logit link function (log odds ratio) and binomial error distribution.
Note: This model is very common as the parameter estimates can be interpreted as the log-odds or back transformed into an odds ratio.

Probit

Fit a model to a binary response variable expressed by the probit function and binomial error distribution.

Related tasks

Fitting a simple logistic regression

Fitting an advanced logistic model

Fitting a simple logistic regression

Fit a simple logistic regression model to describe the relationship between a single predictor variable and a binary response variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Fit Model, and then click Logit / Logistic.
The analysis task pane opens.

3. In the Y drop-down list, select the binary response variable.

4. In the Event drop-down list, select the outcome of interest.

5. In the X drop-down list, select the predictor variable.

6. Click Calculate.

Related concepts

Parameter estimates

Odds ratio estimates

Effect of model hypothesis test

Effect of term hypothesis test

Related tasks

Fitting an advanced logistic model

Estimating odds ratios

Related reference

Dataset layout

Fitting an advanced logistic model

Fit an advanced linear model to describe the relationship between many predictor variables and a binary response variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Fit Model, and then click Logit / Logistic.
The analysis task pane opens.

3. In the Model drop-down menu, select Advanced.

4. In the Y drop-down list, select the response variable.

5. In the Event drop-down list, select the outcome of interest.

6. In the Available variables list, select the predictor variable(s):

- To select a single variable, click the variable.

- To select multiple variables, click the first variable then hold down the CTRL key and click each additional variable.

- To select a range of variables, click the first variable then hold down the SHIFT key and click the last variable in the range.

7. Optional: Click the drop-down menu arrow next to the Available variables list, and then select the measurement scale of the variable.

If the measurement scale is not set, Analyse-it tries to determine if the variable is categorical or continuous scale. After you add a variable to a model, the icon alongside the variable indicates the assumed scale.

8. Click Add to add a variable, or click Factorial to add the factorial terms, or click Polynomial to add a polynomial terms.

9. To add the interaction between two or more variables, in the Terms list box, click the first term then hold down the CTRL key and click each additional term to include in the interaction, and then click Cross.

10. To remove a term, in the Terms list box, click the term and then click Remove.

11. Repeat steps 6 through 10 to build the model.

12. Click Calculate.

Related concepts

Parameter estimates

Odds ratio estimates

Effect of model hypothesis test

Effect of term hypothesis test

Related tasks

Fitting a simple logistic regression

Estimating odds ratios

Related reference

Dataset layout

Fitting a simple probit regression

Fit a simple probit regression model to describe the relationship between a single predictor variable and a binary response variable.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Fit Model, and then click Probit.
The analysis task pane opens.

3. In the Y drop-down list, select the binary response variable.

4. In the Event drop-down list, select the outcome of interest.

5. In the X drop-down list, select the predictor variable.

6. Click Calculate.

Parameter estimates

Parameter estimates (also called coefficients) are associated with a one-unit change of the predictor, all other predictors being held constant.

A coefficient describes the size of the contribution of that predictor; a large coefficient indicates that the variable strongly influences the probability of that outcome, while a near-zero coefficient indicates that variable has little influence on the probability of that outcome. A positive sign indicates that the explanatory variable increases the probability of the outcome, while a negative sign indicates that the variable decreases the probability of that outcome. A confidence interval for each parameter shows the uncertainty in the estimate.

When the model contains categorical variables, the interpretation of the coefficients is more complex. For each term involving a categorical variable, a number of dummy predictor variables are created to predict the effect of each different level. There are different ways to code the predictors for a categorical variable, the most common method in logii/probit regression is called reference cell coding or dummy coding. In reference cell coding, the first category acts as a baseline, and you can interpret the other coefficients as an increase or decrease over the baseline category.

Related concepts

Odds ratio estimates

Point and interval estimation

Related tasks

Estimating odds ratios

Odds ratio estimates

Odds ratios are the increase or decrease in odds associated with a change of the predictor, all other predictors been held constant.

Although mathematically the log odds ratios are easier to work with, when it comes to interpretation, it is more natural to use the odds ratios. Unlike the log odds ratio, the odds ratio is always positive. A value of 1 indicates no change. Values between 0 and less than 1 indicate a decrease in the probability of the outcome event. Values greater than 1 indicate an increase in the probability of the outcome event.

When the predictor variable is a categorical variable, the odds ratio is the increase or decrease in odds over the baseline category.

When a predictor variable is a continuous variable, the odds ratio is the increase or decrease in odds for a change in the predictor variable. The default is for a 1 unit change in the predictor, although it may be more appropriate to use a larger unit, such as for a change of 10 units of the predictor variable.

When a term includes an interaction, the odds ratio will depend on the interacting variables' values, so you will need to use multiple odd ratios to describe the behavior at different levels of interest.

Related concepts

Parameter estimates

Point and interval estimation

Related tasks

Estimating odds ratios

Estimating odds ratios

Estimate the odds ratios using a logistic model.

Ensure you have followed the steps in Fitting a simple logistic regression or Fitting an advanced logistic model.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Fit Model group, click Odds Ratios.
The analysis task pane Odds Ratios panel opens.

3. In the Odds Ratios grid, in the Comparison column.: For a continuous variable, type the change in units of the predictor for the odd ratio, the default is for 1 unit change in the predictor, but for some predictors the odds ratio for a 10 or 100 unit change in the predictor maybe more interpretable. For a categorical variable, select the type of comparison:

Option

Description

All pairs

Compare all pairs of categories against each other. For example, with 3 categories the comparisons, are 1v2,1v3,2v1,2v3,3v1,3v2.

All distinct pairs

Compare all unique pairs of categories against each other. For example, with 3 categories the comparisons as 2v1,3v1,3v2).

Against reference

Compare all categories against the reference category (the first category in the categorical variable). For example, with 3 categories the comparisons are 3v1, 2v1.

4. If the model has interaction terms, in the At column for each continuous variable involved in an interaction, type the levels of interest as a comma (or your list delimiter) separated list. For example, if Age is a variable involved in an interaction with Gender, you may use 16, 25, 30, 40, and 60 as the levels of interest at which the Gender Female/Male odds ratios will be computed. For a continuous variable involved in an interaction with a categorical variable, the unit change for each level of the categorical variable will be computed. For example, if the Age unit is 10 years and Age is involved in an interaction with Gender, the odds ratio for a change of 10 years will be computed for Females and Males.

5. In the Confidence interval edit box, type the confidence level.

6. Click Recalculate.

Related concepts

Parameter estimates

Odds ratio estimates

Effect of model hypothesis test

Effect of term hypothesis test

Effect of model hypothesis test

A likelihood ratio or Wald X² test formally tests the hypothesis of whether the model fits the data better than no model.

It is common to test whether the model fits the data better than the null model with no parameters.

A X^2 test formally tests whether the reduction is statistically significant. The null hypothesis states that all the parameters for the covariates are zero against the alternative that at least one parameter is not equal to zero. When the p-value is small, you can reject the null hypothesis and conclude that at least one parameter is not zero.

Related concepts

Hypothesis testing

Effect of term hypothesis test

A likelihood ratio or Wald X² test formally tests the hypothesis of whether a term contributes to the model.

In most modeling analyses the aim is a model that describes the relationship using as few terms as possible. It is therefore of interest to look at each term in the model to decide if the term is providing any useful information.

A X^2 test for each term is a formal hypothesis test to determine if the term provides useful information to the model.  The null hypothesis states that the term does not contribute to the model, against the alternative hypothesis that it does. When the p-value is small, you can reject the null hypothesis and conclude that the term does contribute to the model.

When a term is not deemed to contribute statistically to the model, you may consider removing it. However, you should be cautious of removing terms that are known to contribute by some underlying mechanism, regardless of the statistical significance of a hypothesis test, and recognize that removing a term can alter the effect of other terms.

Related concepts

Hypothesis testing

Study design

Fit model analysis study requirements and dataset layout.

Requirements

- 1 or more categorical or quantitative predictor variables.

- A categorical or quantitative response variable.

Dataset layout

Use a column for each predictor variable (Height, Sex) and a column for the response variable (Weight); each row has the values of the variables for a case (Subject).

Subject (optional)

Height

Sex

Weight

1

175

M

65

2

180

M

70

3

160

F

90

4

190

F

55

5

180

M

100

6

150

F

55

7

140

M

75

8

160

M

80

9

165

F

80

10

180

M

95

…

…

…

…

Chapter 18. Method comparison / Agreement

Method comparison measures the closeness of agreement between the measured values of two methods.

Note: The term method is used as a generic term and can include different measurement procedures, measurement systems, laboratories, or any other variable that you want to if there are differences between measurements.

- Correlation coefficient
A correlation coefficient measures the association between two methods.

- Scatter plot
A scatter plot shows the relationship between two methods.

- Fit Y on X
Regression of Y on X describes the linear relationship between the methods.

- Linearity
Linearity is the assumption that the relationship between the methods is linear.

- Residual plot
A residual plot shows the difference between the measured values and the predicted values against the true values.

- Average bias
Bias is a measure of a systematic measurement error, the component of measurement error that remains constant in replicate measurements on the same item. When measuring a method against a reference method using many items the average bias is an estimate of bias that is averaged over all the items.

- Difference plot (Bland-Altman plot)
A difference plot shows the differences in measurements between two methods, and any relationship between the differences and true values.

- Fit differences
Regression of the differences on the true value describes the relationship between the methods.

- Limits of agreement (LoA)
Limits of agreement estimate the interval within which a proportion of the differences between measurements lie.

- Mountain plot (folded CDF plot)
A mountain plot shows the distribution of the differences between two methods. It is a complementary plot to the difference plot.

- Agreement measures for binary and semi-quantitative data
Agreement measures summarize the similarity of the results of two binary or semi-quantitative methods.

- Agreement plot
An agreement plot shows the agreement between two binary or semi-quantitative methods.

Correlation coefficient

A correlation coefficient measures the association between two methods.

The correlation coefficient is probably the most commonly reported statistic in method comparison studies. However, it is irrelevant for a number of reasons (Bland & Altman, 1986):

- It is a measure of the strength of linear association between two methods, the extent to which as one variable increases the other variable also tends to increase, not the agreement between them.

- A change in the scale of measurement does not affect the correlation, even though it affects the agreement. For example, if one method reports double the value of another method the correlation coefficient would still be high even though the agreement between the methods is poor.

- It simply represents the ratio of variation between the subjects relative to the measurement variation. The measuring interval chosen in study design can affect the correlation coefficient.

The correlation coefficient is sometimes re-purposed as an adequate range test (CLSI, 2002) on the basis that the ratio of variation between subjects, relative to measurement variation, is an indicator of the quality of the data (Stöckl & Thienpont, 1998). When the correlation coefficient is greater than 0.975 the parameters of an ordinary linear regression are not significantly biased by the error in the X variable, and so linear regression is sometimes recommended. However, with the wide range of proper regression procedures available for analyzing method comparison studies, there is little need to use inappropriate models.

Related concepts

Scatter plot

Fit Y on X

Related information

Bland, J., & Altman, D. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307-310.

Stöckl, D., Dewitte, K., & Thienpont, L. M. (1998). Validity of linear regression in method comparison studies: is it limited by the statistical model or the quality of the analytical input data?. Clinical Chemistry, 44(11), 2340-2346.

CLSI. (2002). Method Comparison and Bias Estimation Using Patient Samples (EP9-A2). Clinical and Laboratory Standards Institute.

Scatter plot

A scatter plot shows the relationship between two methods.

The scatter plot shows measured values of the reference or comparison method on the horizontal axis, against the test method on the vertical axis.

The relationship between the methods may indicate a constant, or proportional bias, and the variability in the measurements across the measuring interval. If the points form a constant-width band, the method has a constant standard deviation (constant SD). If the points form a band that is narrower at small values and wider at large values, there is a constant relationship between the standard deviation and value, and the method has constant a coefficient of variation (CV). Some measurement procedures exhibit constant SD in the low range and constant CV in the high range.

If both methods measure on the same scale, a gray identity line shows ideal agreement and is useful for comparing the relationship against.

Related concepts

Correlation coefficient

Fit Y on X

Fit Y on X

Regression of Y on X describes the linear relationship between the methods.

- Ordinary linear regression
Ordinary linear regression fits a line describing the relationship between two variables assuming the X variable is measured without error.

- Deming regression
Deming regression is an errors-in-variables model that fits a line describing the relationship between two variables. Unlike ordinary linear regression, it is suitable when there is measurement error in both variables.

- Passing-Bablok regression
Passing-Bablok regression fits a line describing the relationship between two variables. It is robust, non-parametric, and is not sensitive to outliers or the distribution of errors.

Related concepts

Correlation coefficient

Scatter plot

Ordinary linear regression

Ordinary linear regression fits a line describing the relationship between two variables assuming the X variable is measured without error.

Ordinary linear regression finds the line of best fit by minimizing the sum of the vertical distances between the measured values and the regression line. It, therefore, assumes that the X variable is measured without error.

Weighted linear regression is similar to ordinary linear regression but weights each item to take account of the fact that some values have more measurement error than others. Typically, for methods that exhibit constant CV the higher values are less precise and so weights equal to 1 / X² are given to each observation.

Although the assumption that there is no error in the X variable is rarely the case in method comparison studies, ordinary and weighted linear regression are often used when the X method is a reference method. In such cases the slope and intercept estimates have little bias, but hypothesis tests and confidence intervals are inaccurate due to an underestimation of the standard errors (Linnet, 1993). We recommend the use of correct methods such as Deming regression.

Related tasks

Fitting ordinary linear regression

Related information

Linnet, K. (1993). Evaluation of regression procedures for methods comparison studies. Clinical Chemistry, 39, 424-424.

Deming regression

Deming regression is an errors-in-variables model that fits a line describing the relationship between two variables. Unlike ordinary linear regression, it is suitable when there is measurement error in both variables.

Deming regression (Cornbleet & Gochman, 1979) finds the line of best fit by minimizing the sum of the distances between the measured values and the regression line, at an angle specified by the variance ratio. It assume both variables are measured with error. The variance ratio of the errors in the X / Y variable is required and assumed to be constant across the measuring interval. If you measure the items in replicate, the measurement error of each method is estimated from the data and the variance ratio calculated.

In the case where the variance ratio is equal to 1, Deming regression is equivalent to orthogonal regression. When only single measurements are made by each method and the ratio of variances is unknown, a variance ratio of 1 is sometimes used as a default. When the range of measurements is large compared to the measurement error this can be acceptable. However in cases where the range is small the estimates are biased and the standard error underestimated, leading to incorrect hypothesis tests and confidence intervals (Linnet, 1998).

Weighted Deming regression (Linnet, 1990) is a modification of Deming regression that assumes the ratio of the coefficient of variation (CV), rather than the ratio of variances, is constant across the measuring interval.

Confidence intervals for parameter estimates use a t-distribution and the standard errors are computed using a Jackknife procedure (Linnet, 1993).

Related tasks

Fitting Deming regression

Related information

Cornbleet, P. J., & Gochman, N. (1979). Incorrect least-squares regression coefficients in method-comparison analysis. Clinical Chemistry, 25(3), 432-438.

Linnet, K. (1990). Estimation of the linear relationship between the measurements of two methods with proportional errors. Statistics in Medicine, 9(12), 1463-1473.

Linnet, K. (1993). Evaluation of regression procedures for methods comparison studies. Clinical Chemistry, 39, 424-424.

Linnet, K. (1998). Performance of Deming regression analysis in case of misspecified analytical error ratio in method comparison studies. Clinical chemistry, 44(5), 1024-1031.

Passing-Bablok regression

Passing-Bablok regression fits a line describing the relationship between two variables. It is robust, non-parametric, and is not sensitive to outliers or the distribution of errors.

Passing-Bablok regression (Passing & Bablok, 1983) finds the line of best fit using a shifted median of all possible pairwise slopes between points. It does not make assumptions of the distribution of the measurement errors, and the variance of the measurement errors need not remain constant over the measuring interval though their ratio should remain proportional to β² (the slope squared, in many cases β ≈ 1).

There is a modification to the procedure for use when the methods measure on different scales, or where the purpose is to transform results from one method to another rather than to compare two methods for equality (Passing & Bablok, 1988).

Confidence intervals for parameter estimates are based on a normal approximation or bootstrap. Confidence curves around the regression line and at specific points on the line are obtained by bootstrap.

Related tasks

Fitting Passing-Bablok regression

Related information

Passing, H., & Bablok, W. (1983). A new biometrical procedure for testing the equality of measurements from two different analytical methods. Application of linear regression procedures for method comparison studies in clinical chemistry, Part I. Clinical Chemistry and Laboratory Medicine, 21(11), 709-720.

Passing, H., & Bablok, W. (1984). Comparison of Several Regression Procedures for Method Comparison Studies and Determination of Sample Sizes Application of linear regression procedures for method comparison studies in Clinical Chemistry, Part II. Clinical Chemistry and Laboratory Medicine, 22(6), 431-445.

Bablok, W., Passing, H., Bender, R., & Schneider, B. (1988). A general regression procedure for method transformation. Application of linear regression procedures for method comparison studies in clinical chemistry, Part III. Clinical Chemistry and Laboratory Medicine, 26(11), 783-790.

Fitting ordinary linear regression

Fit a ordinary least square regression to estimate the relationship between a test method and a reference method when the reference method is measured without error.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Method Comparison, and then click the fit:

Option

Description

Ordinary Least Square

Fit an ordinary regression where the test method measurement error SD is constant throughout the measuring interval.

Weighted Least Squares

Fit an ordinary regression where the test method measurement error CV is constant throughout the measuring interval.

The analysis task pane opens.

3. If the data are in 2 variables:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

Note: If the variables consist of replicate measurements, select the variable name that spans all the replicate columns.

4. If the data are in 2 variables with a separate variable identifying replicates of each item:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

3. In the Item drop-down list, select the variable identifying each item.

5. If the data are in a single variable with a separate variable matching each item and a variable identifying the method:

1. In the Model drop-down menu, select Matched Pairs.

2. In the Y drop-down list, select the measurement variable.

3. In the Item drop-down list, select the item variable that identifies each item.

4. In the Method drop-down list, select the method variable.

6. If the items are measured in replicate, in the Replicates group, select:

Option

Description

1st X, 1st Y

Uses only the 1st X replicate and 1st Y replicate in the regression.

Mean X, 1st Y

Uses the Mean of X replicates and the 1st Y replicate in the regression.

Mean X, Mean Y

Uses the Mean of the X replicates and the Mean of the Y replicates in the regression.

Note: All items must have the same number of replicates. Items that do not are excluded from analysis.

7. Click Calculate.

Related concepts

Ordinary linear regression

Related tasks

Checking the assumptions of the fit

Estimating the bias between methods at a decision level

Partitioning and reducing the measuring interval

Related reference

Dataset layout

Related information

CLSI. (2002). Method Comparison and Bias Estimation Using Patient Samples (EP9-A2). Clinical and Laboratory Standards Institute.

Fitting Deming regression

Fit a Deming regression to estimate the relationship between a test method and a reference or comparative method when both variables are measured with error.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Method Comparison, and then click the fit:

Option

Description

Ordinary Deming

Fit a Deming regression where the ratio of measurement error SD is constant throughout the measuring interval.

Weighted Deming

Fit a Deming regression where the ratio of measurement error CV is constant throughout the measuring interval.

The analysis task pane opens.

3. If the data are in 2 variables:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

Note: If the variables consist of replicate measurements, select the variable name that spans all the replicate columns.

4. If the data are in 2 variables with a separate variable identifying replicates of each item:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

3. In the Item drop-down list, select the variable identifying each item.

5. If the data are in a single variable with a separate variable matching each item and a variable identifying the method:

1. In the Model drop-down menu, select Matched Pairs.

2. In the Y drop-down list, select the measurement variable.

3. In the Item drop-down list, select the item variable that identifies each item.

4. In the Method drop-down list, select the method variable.

6. If the items are measured in replicate, in the Replicates group, select:

Option

Description

1st X, 1st Y

Uses only the 1st X replicate and 1st Y replicate in the regression.

Mean X, 1st Y

Uses the Mean of X replicates and the 1st Y replicate in the regression.

Mean X, Mean Y

Uses the Mean of the X replicates and the Mean of the Y replicates in the regression.

Note: All items must have the same number of replicates. Items that do not are excluded from analysis.

7. If the items are not measured in replicate, in the Variance ratio, edit box, type the ratio of the variances, or in the SD/CV X and SD/CV Y edit boxes, type the standard deviation or CV of the measurement error for each method.

Note: The regression procedure uses all replicate measurements to estimate the precision of each method.

8. Click Calculate.

Related concepts

Deming regression

Related tasks

Testing commutability of other materials

Checking the assumptions of the fit

Estimating the bias between methods at a decision level

Partitioning and reducing the measuring interval

Related reference

Dataset layout

Related information

CLSI. (2002). Method Comparison and Bias Estimation Using Patient Samples (EP9-A2). Clinical and Laboratory Standards Institute.

Fitting Passing-Bablok regression

Fit a Passing-Bablok regression to estimate the relationship between a test method and a reference or comparative method.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Method Comparison, and then click Passing-Bablok.
The analysis task pane opens.

3. If the data are in 2 variables:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

Note: If the variables consist of replicate measurements, select the variable name that spans all the replicate columns.

4. If the data are in 2 variables with a separate variable identifying replicates of each item:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

3. In the Item drop-down list, select the variable identifying each item.

5. If the data are in a single variable with a separate variable matching each item and a variable identifying the method:

1. In the Model drop-down menu, select Matched Pairs.

2. In the Y drop-down list, select the measurement variable.

3. In the Item drop-down list, select the item variable that identifies each item.

4. In the Method drop-down list, select the method variable.

6. If the items are measured in replicate, in the Replicates group, select:

Option

Description

1st X, 1st Y

Uses only the 1st X replicate and 1st Y replicate in the regression.

Mean X, 1st Y

Uses the Mean of X replicates and the 1st Y replicate in the regression.

Mean X, Mean Y

Uses the Mean of the X replicates and the Mean of the Y replicates in the regression.

Note: All items must have the same number of replicates. Items that do not are excluded from analysis.

7. In the Method drop-down list, select:

Option

Description

Part I

When the methods are measured on the same scale and the purpose is to compare if the methods are equal.

Part III

When the methods are measured on different scales, or the purpose is to convert the values from one method to the other using the regression formula.

8. Click Calculate.

Related concepts

Passing-Bablok regression

Related tasks

Checking the assumptions of the fit

Estimating the bias between methods at a decision level

Partitioning and reducing the measuring interval

Related reference

Dataset layout

Related information

CLSI. (2002). Method Comparison and Bias Estimation Using Patient Samples (EP9-A2). Clinical and Laboratory Standards Institute.

Linearity

Linearity is the assumption that the relationship between the methods is linear.

The regression procedures used in method comparison studies assume the relationship between the methods is linear. A CUSUM is a measure of the linearity, defined as a running sum of the number of observations above and below the fitted regression line. When the relationship is linear it is expected the points above and below the line are randomly scattered, and the CUSUM statistic is small. Clusters of points on one side of the regression line produce a large CUSUM statistic.

A formal hypothesis test for linearity is based on the largest CUSUM statistic and the Kolmogorov-Smirnov test. The null hypothesis states that the relationship is linear, against the alternative hypothesis that it is not linear. When the test p-value is small, you can reject the null hypothesis and conclude that the relationship is nonlinear.

Related tasks

Checking the assumptions of the fit

Partitioning and reducing the measuring interval

Residual plot

A residual plot shows the difference between the measured values and the predicted values against the true values.

The residual plot shows disagreement between the data and the fitted model. The ideal residual plot (called the null residual plot) shows a random scatter of points forming an approximately constant width band around the identity line.

It is important to check the fit of the model and the assumptions:

Assumption

How to check

Model function is linear

The points will form a pattern when the model function is not linear.

Constant variance

If the points tend to form an increasing, decreasing, or non-constant width band, the variance is not constant and you should consider using weighted regression.

Normality

A histogram of the residuals should form a normal distribution. This is an assumption of linear regression. Deming regression with Jacknife standard errors is robust to this assumption. Passing-Bablok regression is non-parametric and this assumption does not apply.

Related tasks

Checking the assumptions of the fit

Checking the assumptions of the fit

Inspect the residual and linearity plots to ensure the assumptions of the regression procedure are met before using the analysis results to make inferences.

You must have already completed either of the tasks:

- Fitting ordinary linear regression

- Fitting Deming regression

- Fitting Passing-Bablok regression

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Method Comparison group, click Residuals.

3. On the Analyse-it ribbon tab, in the Method Comparison group, click Linearity.

4. Click Recalculate.

Related concepts

Linearity

Residual plot

Average bias

Bias is a measure of a systematic measurement error, the component of measurement error that remains constant in replicate measurements on the same item. When measuring a method against a reference method using many items the average bias is an estimate of bias that is averaged over all the items.

Bias is the term used when a method is compared against a reference method. When the comparison is not against a reference method but instead another routine comparative laboratory method, it is simply an average difference between methods rather than an average bias. For clarify of writing we will use the term average bias.

The average bias is usually expressed as the constant and proportional bias from a regression procedure, or as a constant or proportional bias from the mean of the differences or relative differences. If there are other sources systematic errors present, such as nonlinearity or interferences, the average bias will be incorrect.

The average bias is an estimate of the true unknown average bias in a single study. If the study were repeated, the estimate would be expected to vary from study to study. Therefore, if a single estimate is compared directly to 0 or compared to the allowable bias the statement is only applicable to the single study. To make inferences about the true unknown bias you must perform a hypothesis test:

There are two common hypotheses of interest that can be tested:

- Equality test
The null hypothesis states that the bias is equal to 0, against the alternative hypothesis that it is not equal zero. When the test p-value is small, you can reject the null hypothesis and conclude that the bias is different to zero.

It is important to remember that a statistically significant p-value tells you nothing about the practical importance of what was observed. For a large sample, the bias for a statistically significant hypothesis test may be so small as to be practically useless. Conversely, although there may some evidence of bias, the sample size may be too small for the test to reach statistical significance, and you may miss an opportunity to discover a true meaningful bias. Lack of evidence against the null hypothesis does not mean it has been proven to be true, the belief before you perform the study is that the null hypothesis is true and the purpose is to look for evidence against it. An equality test at the 5% significance level is equivalent to testing if the 95% confidence interval includes zero.

- Equivalence test
The null hypothesis states that the bias is outside an interval of practical equivalence, against the alternative hypothesis that the bias is within the interval considered practically equivalent. When the test p-value is small, you can reject the null hypothesis and conclude that the bias is practically equivalent, and within the specified interval.

An equivalence test is used to prove a bias requirement can be met. The null hypothesis states the methods are not equivalent and looks for evidence that they are in fact equivalent. An equivalence hypothesis test at the 5% significance level is the same as testing if the 90% confidence interval lies within the allowable bias interval.

Related tasks

Estimating the bias between methods at a decision level

Estimating the bias between methods at a decision level

Estimate the average bias (or average difference) at a decision level using the regression fit.

You must have already completed either of the tasks:

- Fitting ordinary linear regression

- Fitting Deming regression

- Fitting Passing-Bablok regression

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Method Comparison group, click Predict At.
The analysis task pane shows the Comparability task.

3. In the Decision level grid, under the Id column, type an identifier for the decision level, and then under the X column, type the value of the decision level

4. Optional: To test if the two methods are equal at each decision level:

1. On the Analyse-it ribbon tab, in the Method Comparison group, click Test Equality.
The analysis task pane shows the Comparability task.

2. To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

5. Optional: To test if two methods are practically equivalent at each decision level:

1. On the Analyse-it ribbon tab, in the Method Comparison group, click Test Equivalence.
The analysis task pane shows the Comparability task.

2. If the allowable difference is a constant or proportional value across the measuring interval, in the Allowable difference group, select Across measuring interval, and then in the Absolute edit box, type the bias in measurement units, and/or in the Relative edit box, type the bias as a percentage (suffix with % symbol).

3. If the allowable difference varies for each level, in the Allowable difference group, select Each level and then in the Decision level grid, under the Allowable difference column, alongside each level, type the absolute bias in measurement units, or the relative bias as a percentage (suffix with % symbol).

4. To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

6. Click Recalculate.

Related concepts

Average bias

Testing commutability of other materials

Test the commutability of reference materials or processed samples with unprocessed samples.

You must fit a Deming regression to perform this task.

1. On the Analyse-it ribbon tab, in the Method Comparison group, click Restrict to Group.

2. In the Group / Color / Symbol drop-down list, select the variable indicating the type of sample.

3. In the Restrict fit to group drop-down list, select the group of unprocessed samples.

4. On the Fit Y on X task pane, in the Prediction band edit box, type 95%, and if required select the Familywise coverage check box.

5. Click Recalculate.

The scatter plot shows the points in different colors depending on the type of sample. The fit is restricted to a unprocessed samples and the prediction band includes any additional samples that are commutable with the unprocessed samples. A table lists the additional samples and if they are commutable with the unprocessed samples.

Related tasks

Fitting Deming regression

Difference plot (Bland-Altman plot)

A difference plot shows the differences in measurements between two methods, and any relationship between the differences and true values.

Bland and Altman (1983) popularized the use of the difference plot for assessing agreement between two methods. Although their use was to show the limits of agreement between two methods, the plot is also widely used for assessing the average bias (average difference) between two methods.

The classic difference plot shows the difference between the methods on the vertical axis, against the best estimate of the true value on the horizontal axis. When one method is a reference method, it is used as the best estimate of the true value and plotted on the horizontal axis (Krouwer, 2008). In other cases, using the average of the methods as the best estimate of the true value, to avoid an artificial relationship between the difference and magnitude (Bland & Altman, 1995).

A relative difference plot (Pollock et al., 1993) shows the relative differences on the vertical axis, against the best estimate of the true value on the horizontal axis. It is useful when the methods show variability related to increasing magnitude, that is where the points on a difference plot form a band starting narrow and becoming wider as X increases. Another alternative is to plot the ratio of the methods on the vertical axis, against the best estimate of the true value on the horizontal axis.

Related concepts

Fit differences

Limits of agreement (LoA)

Related information

Altman, D. G., & Bland, J. M. (1983). Measurement in medicine: the analysis of method comparison studies. The statistician, 307-317.

Bland, J., & Altman, D. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307-310.

Bland, J. M., & Altman, D. G. (1999). Measuring agreement in method comparison studies. Statistical methods in medical research, 8(2), 135-160.

Carstensen, B. (2011). Comparing clinical measurement methods: a practical guide. John Wiley & Sons.

Pollock, M. A., Jefferson, S. G., Kane, J. W., Lomax, K., MacKinnon, G., & Winnard, C. B. (1992). Method comparison - a different approach. Annals of clinical biochemistry, 29, 556-560.

Bland, J. M., & Altman, D. G. (1995). Comparing methods of measurement: why plotting difference against standard method is misleading. The Lancet, 346(8982), 1085-1087.

Krouwer, J. S. (2008). Why Bland–Altman plots should use X, not (Y+ X)/2 when X is a reference method. Statistics in Medicine, 27(5), 778-780.

Fit differences

Regression of the differences on the true value describes the relationship between the methods.

Mean difference measures the constant relationship between the variables. An assumption is that the difference is not relative to magnitude across the measuring interval. If the differences are related to the magnitude, the relationship should be modeled by using relative differences or regressing the differences on the true value.

Related concepts

Difference plot (Bland-Altman plot)

Limits of agreement (LoA)

Related tasks

Plotting a difference plot and estimating the average bias

Plotting a difference plot and estimating the average bias

Plot a difference plot and estimate the average bias (average difference) between the methods.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Method Comparison, and then click:

Option

Description

Mean difference

Estimate the average difference using the mean of the differences.

Median difference

Estimate the average difference using the median of the differences. Useful when the distribution of the differences is not normally distributed

The analysis task pane opens.

3. If the data are in 2 variables:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

Note: If the variables consist of replicate measurements, select the variable name that spans all the replicate columns.

4. If the data are in 2 variables with a separate variable identifying replicates of each item:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

3. In the Item drop-down list, select the variable identifying each item.

5. If the data are in a single variable with a separate variable matching each item and a variable identifying the method:

1. In the Model drop-down menu, select Matched Pairs.

2. In the Y drop-down list, select the measurement variable.

3. In the Item drop-down list, select the item variable that identifies each item.

4. In the Method drop-down list, select the method variable.

6. In the D drop-down list, select:

Option

Description

Difference

The differences are not related to magnitude.

Relative Difference

The differences are related to magnitude.

7. In the Z drop-down list, select:

Option

Description

X

X is a reference method.

(X+Y)/2

Neither X or Y are reference methods.

8. Click Calculate.

Related concepts

Fit differences

Related tasks

Partitioning and reducing the measuring interval

Related reference

Dataset layout

Related information

CLSI. (2002). Method Comparison and Bias Estimation Using Patient Samples (EP9-A2). Clinical and Laboratory Standards Institute.

Limits of agreement (LoA)

Limits of agreement estimate the interval within which a proportion of the differences between measurements lie.

The limits of agreement includes both systematic (bias) and random error (precision), and provide a useful measure for comparing the likely differences between individual results measured by two methods. When one of the methods is a reference method, the limits of agreement can be used as a measure of the total error of a measurement procedure (Krouwer, 2002).

Limits of agreement can be derived using parametric method given the assumption of normality of the differences. or using non-parametric percentiles when such assumptions do not hold.

Related concepts

Difference plot (Bland-Altman plot)

Fit differences

Related tasks

Plotting the Bland-Altman limits of agreement

Related information

Altman, D. G., & Bland, J. M. (1983). Measurement in medicine: the analysis of method comparison studies. The statistician, 307-317.

Bland, J., & Altman, D. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307-310.

Bland, J. M., & Altman, D. G. (1999). Measuring agreement in method comparison studies. Statistical methods in medical research, 8(2), 135-160.

Carstensen, B. (2011). Comparing clinical measurement methods: a practical guide. John Wiley & Sons.

Krouwer, J. S. (2002). Setting performance goals and evaluating total analytical error for diagnostic assays. Clinical Chemistry, 48(6), 919-927.

Plotting the Bland-Altman limits of agreement

Plot a difference plot and limits of agreement.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Method Comparison(or Agreement depending on the edition), and then click Bland-Altman.
The analysis task pane opens.

3. If the data are in 2 variables:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

Note: If the variables consist of replicate measurements, select the variable name that spans all the replicate columns.

4. If the data are in 2 variables with a separate variable identifying replicates of each item:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

3. In the Item drop-down list, select the variable identifying each item.

5. If the data are in a single variable with a separate variable matching each item and a variable identifying the method:

1. In the Model drop-down menu, select Matched Pairs.

2. In the Y drop-down list, select the measurement variable.

3. In the Item drop-down list, select the item variable that identifies each item.

4. In the Method drop-down list, select the method variable.

6. In the D drop-down list, select:

Option

Description

Difference

Plot the differences on the vertical axis.

Relative Difference

Plot the differences relative to Z on the vertical axis.

Ratio

Plot the ratio of the values on the vertical axis.

7. In the on Z drop-down list, select:

Option

Description

X

Plot X as a reference method on the horizontal axis.

(X+Y)/2

Plot the average on the horizontal axis. Neither X or Y are reference methods.

8. In the Mean function drop-down list, select:

Option

Description

Constant

There is a constant relationship between D and Z.

Linear function

There is a linear relationship between D and Z.

9. In the SD drop-down list, select:

Option

Description

Constant

The measurement error is constant throughout the measuring interval.

Linear function

The measurement error is linear across the measuring interval.

10. Click Calculate.

Related concepts

Limits of agreement (LoA)

Related tasks

Partitioning and reducing the measuring interval

Related reference

Dataset layout

Related information

CLSI. (2013). Estimation of Total Analytical Error for Clinical Laboratory Methods (EP21-A). Clinical and Laboratory Standards Institute.

Mountain plot (folded CDF plot)

A mountain plot shows the distribution of the differences between two methods. It is a complementary plot to the difference plot.

Krouwer and Monti (1995) devised the mountain plot (also known as a folded empirical cumulative distribution plot) as a complementary representation of the difference plot. It shows the distribution of the differences with an emphasis on the center and the tails of the distribution. You can use the plot to estimate the median of the differences, the central 95% interval, the range, and the percentage of observations outside the total allowable error bands.

The plot is simply the empirical cumulative distribution function of the differences folded around the median (that is, the plotted function = p where p < 0.5 otherwise 1-p). Unlike the histogram it is unaffected by choice of class intervals, however it should be noted that although the mountain plot looks like a frequency polygon it does not display the density function. It has recently been proven that the area under the plot is equal to the mean absolute deviation from the median (Xue and Titterington, 2010).

Related tasks

Plotting a mountain plot

Related information

Krouwer, J. S., & Monti, K. L. (1995). A simple, graphical method to evaluate laboratory assays. European journal of clinical chemistry and clinical biochemistry, 33(8), 525-528.

Monti, K. L. (1995). Folded empirical distribution function curves—mountain plots. The American Statistician, 49(4), 342-345.

Xue J., Titterington D. M. (2010, unpublished). The p-folded Cumulative Distribution Function and the Mean Absolute Deviation from the p-quantile.

Plotting a mountain plot

Plot a mountain plot to see the distribution of the differences between two methods.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Method Comparison (or Agreement depending on the edition), and then click Mountain.
The analysis task pane opens.

3. If the data are in 2 variables:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

Note: If the variables consist of replicate measurements, select the variable name that spans all the replicate columns.

4. If the data are in 2 variables with a separate variable identifying replicates of each item:

1. In the X drop-down list, select the comparative or reference measurement procedure variable.

2. In the Y drop-down list, select the test measurement procedure variable.

3. In the Item drop-down list, select the variable identifying each item.

5. If the data are in a single variable with a separate variable matching each item and a variable identifying the method:

1. In the Model drop-down menu, select Matched Pairs.

2. In the Y drop-down list, select the measurement variable.

3. In the Item drop-down list, select the item variable that identifies each item.

4. In the Method drop-down list, select the method variable.

6. If the data are measured in replicate and the X method is a reference method and the Y method is a comparative method, select the Mean X, 1st Y replicate check box so that the differences represent the difference between an individual test result by the Y method and the average of the replicates for the X method. These differences are the best representation of the total error in the test method, by using the mean of the X method it reduces the amount of random error in the X result so it reflects the true value of the reference method to compare the test method against. If you select Mean X, Mean Y, the differences will only represent the some of the error, namely the systematic error due to bias with some of the random error removed due to averaging of replicates. Likewise, if you select 1st X, replicate, 1st Y replicate the differences will include any random error in the X method so not reflect the error just in the test method. There may be occasions when these other sources of error are of interest, but generally the interest is in the total error of the test method.

7. Click Calculate.

Related concepts

Mountain plot (folded CDF plot)

Related reference

Dataset layout

Related information

CLSI. (2013). Estimation of Total Analytical Error for Clinical Laboratory Methods (EP21-A). Clinical and Laboratory Standards Institute.

Partitioning and reducing the measuring interval

Partition or reduce the measuring interval to fit different functions.

You may need to reduce the measuring interval if a shows nonlinearity at the ends of the measuring interval. Or, you may need to partition the measuring interval if the differences show constant relationship at small magnitude and proportional relationship at higher values.

1. Activate the analysis report worksheet.

2. To reduce the range of the measuring interval, On the Analyse-it ribbon tab, in the Method Comparison group, click Adjust Interval.
The analysis task pane shows the Comparability task.

3. To partition the measuring interval, On the Analyse-it ribbon tab, in the Method Comparison group, click Partition.

4. In the Measuring interval(s) grid, under either or both the Lower and Upper column, in the edit boxes, type the lower and upper measuring interval or partition.

Note: If you edit the upper/lower limit of a partition, the lower/upper limit of the next partition is set to the same value respectively.

5. To only plot points in each interval on the analysis reports, clear the Show data points outside interval check box. If you select this option, the points are shaded gray on plots.

6. Change the options on the tasks.

Note: Each partition has its own task settings. If you want to use the same settings for a number of partitions, it is best to configure the settings before you partition the measuring interval.

7. Repeat steps 3 through 6 for each partition.

8. Click Recalculate.

The analysis creates a separate analysis report for each partition. You can quickly switch between analysis reports for each partition by selecting the partition in the partition grid.

Related concepts

Linearity

Agreement measures for binary and semi-quantitative data

Agreement measures summarize the similarity of the results of two binary or semi-quantitative methods.

Asymmetric agreement measures are dependent on the assignment of X and Y variables. They are often useful due to the natural interpretation as the proportion of the comparative method (X) results in which the new method (Y) results are the same.

For a binary test, with positive/negative results, the results can be expressed as a 2x2 contingency table:

The positive agreement (PPA) and negative agreement (NPA) have a natural interpretation:

- Positive agreement is the proportion of comparative/reference method positive results in which the test method result is positive.

- Negative agreement is the proportion of comparative/reference method negative results in which the test method result is negative.

Symmetric agreement measures are not affected by interchanging the X and Y variable. These are useful in many other cases, such as comparing observers, laboratories, or other factors where neither is a natural comparator. There are various measures based on the mean of the proportions to which X agrees with Y, and Y agrees with X. The Kulczynski, Dice-Sørensen, and Ochiai are three such measures that use arithmetic, harmonic, and geometric mean of the proportions, respectively.

The harmonic mean weights the smaller proportion more heavily and produces the smallest value amongst the three measures. The geometric mean is the square root of Bangdiwala’s B statistic, which is the ratio of the observed agreement to maximum possible agreement in the agreement plot. The arithmetic mean has the greatest value. In most cases, of moderate to high agreement, there is very little to choose between the measures.

The average positive agreement and average negative agreement using the Dice-Sørensen measure are:

- Average positive agreement is the number of positive matches as a proportion of the average of the number of positive results by X, Y.

- Average negative agreement is the number of negative matches as a proportion of the average of the number of negative results by X, Y.

The overall proportion of agreement is the sum of the diagonal entries divided by the total.

Related concepts

Agreement plot

Chance corrected agreement measures for binary and semi-quantitative data

Related tasks

Estimating agreement between two binary or semi-quantitative methods

Related information

Warrens, M. J. (2019). Similarity measures for 2× 2 tables. Journal of Intelligent & Fuzzy Systems, 36(4), 3005-3018.

Chance corrected agreement measures for binary and semi-quantitative data

A chance corrected agreement measure takes into account the possibility of agreement occurring by chance.

The Kappa coefficient is the most popular measure for chance corrected agreement between qualitative variables. It is the overall observed agreement corrected for the possibility of agreement occurring by chance. A weighted version of the statistic is useful for ordinal variables as it weights disagreements dependent on the degree of disagreement between observers. As the Kappa coefficient is an overall summary statistic, it should be accompanied by an agreement plot which can show more insight than an overall summary statistic.

Interpretation of the Kappa coefficient is difficult. The most popular set of criteria for assessing agreement Landis and Koch (1977), who characterized values < 0 as indicating no agreement and 0–0.20 as slight, 0.21–0.40 as fair, 0.41–0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1 as almost perfect agreement. This set of guidelines is, however, by no means universally accepted. A substantial imbalance in the contingency table's marginal totals, either horizontally or vertically, results in a lower Kappa coefficient. And, it will be higher if the imbalance in the corresponding marginal totals is asymmetrical rather than symmetrical, or imperfectly rather than perfectly symmetrical.

Related concepts

Agreement plot

Agreement measures for binary and semi-quantitative data

Related tasks

Estimating agreement between two binary or semi-quantitative methods

Related information

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. biometrics, 159-174.

Feinstein, A. R., & Cicchetti, D. V. (1990). High agreement but low kappa: I. The problems of two paradoxes. Journal of clinical epidemiology, 43(6), 543-549.

Cicchetti, D. V., & Feinstein, A. R. (1990). High agreement but low kappa: II. Resolving the paradoxes. Journal of clinical epidemiology, 43(6), 551-558.

Agreement plot

An agreement plot shows the agreement between two binary or semi-quantitative methods.

Bangdiwala (2013) devised the agreement plot as a complement to the kappa or B-statistics. It is invaluable for assessing agreement as it gives a visual impression that no summary statistic can convey.

The agreement plot is a visual representation of a k by k square contingency table. Each black rectangle represents the marginal totals of the rows and columns. Shaded boxes represent the agreement based on the diagonal cell frequencies; they are positioned inside the rectangles using the sum of the off-diagonal cell frequencies from the same row and column. The partial agreement in the off-diagonal cells can be represented similarly with decreased shading based on the distance from the diagonal. The visualization is affected by the order of the categories, and so the plot is only useful for ordinal or binary data. The plot can have the origin at the bottom-left corner, or at the top-left where it more clearly mimics the contingency table.

Perfect agreement is represented by rectangles that are all perfect squares, with corners on the diagonal identity line, and with shaded boxes equal to the rectangle. Lesser agreement is represented by the area of shaded boxes compared to the area of rectangles. The path of the rectangles, how they deviate from the 45-degree identity line, represents bias in the marginal totals.

Related concepts

Agreement measures for binary and semi-quantitative data

Chance corrected agreement measures for binary and semi-quantitative data

Related tasks

Estimating agreement between two binary or semi-quantitative methods

Related information

Bangdiwala, S. I., & Shankar, V. (2013). The agreement chart. BMC medical research methodology, 13(1), 97.

Estimating agreement between two binary or semi-quantitative methods

Measure the agreement between a test method and a reference or comparative method when both variables are binary or semi-quantitative.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Method Comparison (or Agreement depending on the edition), and then click Binary (PPA/NPA) or Semi-Quantitative.
The analysis task pane opens.

3. In the X drop-down list, select the comparative or reference measurement procedure, or the first observer/factor variable.

4. In the Y drop-down list, select the test measurement procedure, or the second observer/factor variable.

5. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

6. In the Measure drop-down list, select the measure of agreement:

Option

Description

Reference

Compute the proportion of the Y (Test method) that agree with the X (Reference / Comparative method). Useful to make comparisons against a reference method or the current laboratory method. Note: This method is asymmetric and the agreement is different depending on the assignment of X and Y.

Average

Compute the Dice-Sorenson measure of average agreement. Useful when comparing say 2 observers or laboratories where neither is a natural comparator. Note: This method is symmetric and doesn't depend on assignment of X and Y.

7. On the Analyse-it ribbon tab, in the Method Comparison group, click Estimate and then select an overall agreement estimator.

8. Click Calculate.

Related concepts

Agreement plot

Agreement measures for binary and semi-quantitative data

Chance corrected agreement measures for binary and semi-quantitative data

Related reference

Study design for qualitative methods

Study design

Method comparison study requirements and dataset layout.

Requirements

- 2 quantitative variables.

- A recommended minimum of at least 40 cases.

- Each measurement in singlicate or replicate.

Paired dataset layout

Use a column for each method (Method 1, Method 2); each row has the measurement by each method for an item (Subject).

Subject (optional)

Method 1

Method 2

1

120

121

2

113

118

3

167

150

4

185

181

5

122

122

…

…

…

Paired dataset layout with replicates

Use multiple columns for the replicates of each method (Method 1, Method 2); each row has the replicate measurements by each method for an item (Subject).

Subject (optional)

Method 1

Method 2

1

120

122

121

120

2

113

110

118

119

3

167

170

150

155

4

185

188

181

180

5

122

123

122

130

…

…

…

…

…

Paired dataset layout with matched replicates

Use a column for each method (Method 1, Method 2); each row has the measurements by each methods for an item (Subject), with multiple rows for replicates of each item.

Subject

Method 1

Method 2

1

120

121

1

122

120

2

113

118

2

110

119

3

167

150

3

170

155

4

185

181

4

188

180

5

122

122

5

123

130

…

…

…

Matched dataset layout

Use a column for the measurement variable (Measured value), a column for the method variable (Method), and a column for the item variable (Subject); each row has the measurement for a method for an item (Subject), with multiple rows for replicates of each item.

Subject

Method

Measured value

1

X

120

1

X

122

1

Y

121

1

Y

120

2

X

113

2

X

110

2

Y

118

2

Y

119

3

X

167

3

X

170

3

Y

150

3

Y

155

4

X

185

4

X

188

4

Y

181

4

Y

180

5

X

122

5

X

123

5

Y

122

5

Y

130

…

…

…

Study design for qualitative methods

Method comparison study requirements and dataset layout for qualitative methods.

Requirements

- 2 qualitative (binary / semi-quantitative) variables.

Dataset layout for a two qualitative tests

Use a column for the reference/comparative method, and a column for the test method; each row has the values of the variables for a case (Subject).

Subject (optional)

Reference/Comparative method

Test method

1

+

+

2

-

-

3

+

+

4

-

-

5

-

-

6

-

-

7

+

+

8

+

-

9

-

-

10

-

-

…

…

…

Frequency form dataset layout for two qualitative tests

Use a column for the reference/comparative method, and a column for the test method and a column for the number of cases (Frequency); each row has the values of the variables and the frequency count.

Reference / Comparative method

Test method

Count

+

+

32

+

-

1

-

+

3

-

-

38

Chapter 19. Measurement systems analysis (MSA)

Measurement systems analysis is the study of a measurement system to ensure it can meet requirements.

- Precision
Precision is the closeness of agreement between measured quantity values obtained by replicate measurements on the same or similar objects under specified conditions.

- Trueness
Trueness is the closeness of agreement between the average of an infinite number of replicate measured quantity values and a reference quantity value.

- Detection capability
Detection capability describes the performance of a measurement system at small measured quantity values.

Precision

Precision is the closeness of agreement between measured quantity values obtained by replicate measurements on the same or similar objects under specified conditions.

Precision is not a quantity and therefore it is not expressed numerically. Rather, it is expressed by measures such as the variance, standard deviation, or coefficient of variation under the specified conditions of measurement

- Conditions of measurement
Many different factors may contribute to the variability between replicate measurements, including the operator; the equipment used; the calibration of the equipment; the environment; the time elapsed between measurements.

- Variance components
Variance components are estimates of a part of the total variability accounted for by a specified source of variability.

- Precision profile plot
A precision profile plot shows precision against the measured quantity value over the measuring interval.

- Variance function
A variance function describes the relationship between the variance and the measured quantity value.

Conditions of measurement

Many different factors may contribute to the variability between replicate measurements, including the operator; the equipment used; the calibration of the equipment; the environment; the time elapsed between measurements.

Two conditions of precision, termed repeatability and reproducibility conditions are useful for describing the variability of a measurement procedure. Other intermediate conditions between these two extreme conditions of precision are also conceivable and useful, such as conditions within a single laboratory.

- Reproducibility
Reproducibility is a measure of precision under a defined set of conditions: different locations, operators, measuring systems, and replicate measurements on the same or similar objects.

- Intermediate precision
Intermediate precision (also called within-laboratory or within-device) is a measure of precision under a defined set of conditions: same measurement procedure, same measuring system, same location, and replicate measurements on the same or similar objects over an extended period of time. It may include changes to other conditions such as new calibrations, operators, or reagent lots.

- Repeatability
Repeatability (also called within-run precision) is a measure of precision under a defined set of conditions: same measurement procedure, same operators, same measuring system, same operating conditions and same location, and replicate measurements on the same or similar objects over a short period of time.

Reproducibility

Reproducibility is a measure of precision under a defined set of conditions: different locations, operators, measuring systems, and replicate measurements on the same or similar objects.

Related concepts

Conditions of measurement

Intermediate precision

Intermediate precision (also called within-laboratory or within-device) is a measure of precision under a defined set of conditions: same measurement procedure, same measuring system, same location, and replicate measurements on the same or similar objects over an extended period of time. It may include changes to other conditions such as new calibrations, operators, or reagent lots.

Related concepts

Conditions of measurement

Repeatability

Repeatability (also called within-run precision) is a measure of precision under a defined set of conditions: same measurement procedure, same operators, same measuring system, same operating conditions and same location, and replicate measurements on the same or similar objects over a short period of time.

Related concepts

Conditions of measurement

Variance components

Variance components are estimates of a part of the total variability accounted for by a specified source of variability.

Random factors are factors where a number of levels are randomly sampled from the population, and the intention is to make inferences about the population. For example, a study might examine the precision of a measurement procedure in different laboratories. In this case, there are 2 variance components: variation within an individual laboratory and the variation among all laboratories. When performing the study it is impractical to study all laboratories, so instead a random sample of laboratories are used. More complex studies may examine the precision within a single run, within a single laboratory, and across laboratories.

Most precision studies use a nested (or hierarchical) model where each level of a nested factor is unique amongst each level of the outer factor. The basis for estimating the variance components is the nested analysis of variance (ANOVA). Estimates of the variance components are extracted from the ANOVA by equating the mean squares to the expected mean squares. If the variance is negative, usually due to a small sample size, it is set to zero. Variance components are combined by summing them to estimate the precision under different conditions of measurements.

The variance components can be expressed as a variance, standard deviation (SD), or coefficient of variation (CV). A point estimate is a single value that is the best estimate of the true unknown parameter; a confidence interval is a range of values and indicates the uncertainty of the estimate. A larger estimate reflects less precision.

There are numerous methods for constructing confidence intervals for variance components:

Estimator

Description

Exact

Used to form intervals on the inner-most nested variance components. Based on the F-distribution.

Satterthwaite

Used to form intervals on the sum of the variances. Based on the F-distribution, as above, but uses modified degrees of freedom. Works well when the factors have equal or a large number of levels, though when the differences between them are large it can produce unacceptably liberal confidence intervals.

Modified Large Sample (MLS)

Used to form intervals on the sum of variances or individual components. A modification of the large sample normal theory approach to constructing confidence intervals. Provides good coverage close to the nominal level in a wide range of cases.

Related tasks

Estimating the precision of a measurement system

Related information

Burdick, R. K. & Graybill, F. A. (1992). Confidence intervals on variance components. CRC Press.

Sahai, H., & Ojeda, M. M. (2004). Analysis of Variance for Random Models, Volume 1: Balanced Data. Springer Science & Business Media.

Sahai, H., & Ojeda, M. M. (2004). Analysis of Variance for Random Models, Volume 2: Unbalanced Data. Springer Science & Business Media.

ISO. (1994). ISO 5725: Accuracy (Trueness and Precision) of Measurement Methods and Results. International Organization for Standardization.

Estimating the precision of a measurement system

Estimate the precision of a measurement system or procedure.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Precision, and then click:

Option

Description

Simple

Measurements are made under the same conditions.

1 Factor

Measurements vary by a single random factor.
Use for performing CLSI EP5 or EP15 in a single laboratory with Run as a factor.

2 Factor Nested

Measurements vary by 2 random factors.
Use for performing CLSI EP5 in a single laboratory with Day and Run as nested factors.

3 Factor Nested

Measurements vary by 3 random factors.
Use for performing CLSI EP5 in multiple laboratories with Laboratory, Day, Run as nested factors.

The analysis task pane opens.

3. In the Y drop-down list, select the measured quantity variable.

4. If required: In the Factor A drop-down list, select the first random factor.

5. If required: In the Factor B nested within Factor A drop-down list, select the second random factor that is nested within the first factor.

6. If required: In the Factor C nested within Factor B drop-down list, select the third random factor that is nested within the second factor.

7. Optional: If there are multiple levels of measurement, in the By drop-down list, select the level variable.

8. Optional: To label the conditions of measurement for the innermost and outermost nested factors, in the Conditions drop-down list, select the labels.

9. Optional: To customize the detail of the output, select or clear the Abbreviated and Detail check boxes as appropriate.

10. Optional: To compare the precision against performance requirements:

- If the allowable imprecision is a constant or proportional value across the measuring interval, under the Allowable imprecision group, select Across measuring interval, and then in the SD / CV edit box, type the SD in measurement units, or the CV as a percentage (suffix with % symbol).

- If the allowable imprecision varies for each level, under the Allowable imprecision group, select Each level and then in the Allowable imprecision grid, under the SD / CV column, alongside each level, type the SD in measurement units, or the CV as a percentage (suffix with % symbol).

Note: To compare precision against a performance claim, see Testing precision against a performance claim.

11. Click Calculate.

Related concepts

Precision

Variance components

Related tasks

Testing precision against a performance claim

Related reference

Dataset layout

Related information

CLSI. (2014a). Evaluation of Precision Performance of Quantitative Measurement Methods (EP5-A3). Clinical and Laboratory Standards Institute.

CLSI. (2014b). User Verification of Performance for Precision and Trueness (EP15-A3). Clinical and Laboratory Standards Institute.

ISO. (1994). ISO 5725: Accuracy (Trueness and Precision) of Measurement Methods and Results. International Organization for Standardization.

Testing precision against a performance claim

Test if the precision matches a performance claim.

You must have already completed the task:

- Estimating the precision of a measurement system

You should use this procedure when you already have a performance claim from a manufacturer's package insert, and you want to test whether your precision is significantly greater than the claim. It is possible for the precision from a study to be greater than the claim, but for the difference to be due to the random error in your study.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the MSA group, click Test.
The analysis task pane Precision panel opens.

3. In the Hypothesized imprecision grid, alongside each level, enter the performance claim as the SD in measurement units or the CV as a percentage (suffix with % symbol).

4. Optional: In the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%) and if required select the Familywise error rate check box.

Note: CLSI EP15 uses a 5% familywise significance level so the overall significance level is a maximum of 5% regardless of the number of comparisons (for example, for 1 level the significance level is 5%, for 2 levels the significance level is 5%/2 = 2.5% for each level, for 3 levels the significance level is 5%/3 = 1.6% for each level).

5. Click Recalculate.

Related concepts

Precision

Related information

CLSI. (2014b). User Verification of Performance for Precision and Trueness (EP15-A3). Clinical and Laboratory Standards Institute.

Precision profile plot

A precision profile plot shows precision against the measured quantity value over the measuring interval.

Related concepts

Variance function

Related tasks

Plotting a precision profile

Fitting a variance function

Related information

Strike, P. W. (2014). Statistical methods in laboratory medicine. Butterworth-Heinemann.

Plotting a precision profile

Plot a precision profile to observe the performance over the entire measuring interval.

You must have already completed the task:

- Estimating the precision of a measurement system

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the MSA group, click Profile.
The analysis task pane Precision Profile panel opens.

3. In the Precision profile drop-down list, select the measure of precision to plot.

4. Optional: Select the required variance component check boxes.

5. Optional: If the measurements spread over a wide interval with many concentrated near zero, select the Logarithmic X-axis scale check box, so they are more clearly visible.

6. Click Recalculate.

Related concepts

Precision profile plot

Related tasks

Fitting a variance function

Variance function

A variance function describes the relationship between the variance and the measured quantity value.

The are numerous models that describe the relationship between the variance and measured quantity value across the measuring interval:

Fit

Description

Constant variance

Fit constant variance across the measuring interval.

Constant CV

Fit constant coefficient of variation across the measuring interval.

Mixed constant / proportional variance

Fit constant variance at low levels with constant coefficient of variation at high levels.

2-parameter

Fir a 2-parameter linear variance function.

3-parameter

Fit Sadler 3-parameter variance function – a monotone relationship (either increasing or decreasing) between the variance and level of measurement.

3-parameter alternative

Fit Sadler alternate 3-parameter variance function – gives more flexibility than the Sadler standard 3-parameter variance function especially near zero.

4-parameter

Fit a Sadler 4-parameter variance function - allows for a turning point near the detection limit.

A variance function can be useful to estimate the limit of detection or limit of quantitation.

Related concepts

Precision profile plot

Related tasks

Fitting a variance function

Related information

Sadler, W. A., & Smith, M. H. (1986). A reliable method of estimating the variance function in immunoassay. Computational Statistics & Data Analysis, 3, 227-239.

Sadler, W. A., Smith, M. H., & Legge, H. M. (1988). A method for direct estimation of imprecision profiles, with reference to immunoassay data. Clinical chemistry, 34(6), 1058-1061.

Sadler, W. A., (2016). Using the variance function to estimate limit of blank, limit of detection and their confidence intervals. Ann Clin Biochem, 53, 141-149.

Fitting a variance function

Plot a precision profile to observe the performance over the entire measuring interval.

You must have already completed the task:

- Estimating the precision of a measurement system

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the MSA group, click Profile.
The analysis task pane Precision Profile panel opens.

3. Select the Fit variance function check box.

4. In the Model drop-down list, select the variance function.

5. Optional: To estimate the limit of quantitation using functional sensitivity, select the Predict X given CV check box, and then in the edit box, type the CV as a percentage (for example 20%).

6. Click Recalculate.

Related concepts

Variance function

Precision profile plot

Trueness

Trueness is the closeness of agreement between the average of an infinite number of replicate measured quantity values and a reference quantity value.

Trueness is not a quantity and therefore cannot be expressed numerically. Rather it is expressed as bias.

- Bias
Bias is a measure of a systematic measurement error, the component of measurement error that remains constant in replicate measurements.

- Linearity
Nonlinear bias is a component of bias that cannot be represented by a linear relationship between the measured and true values.

- Interferences
Interference bias is a component of bias caused by nonspecificity attributable to the presence of a specific interfering substance.

Bias

Bias is a measure of a systematic measurement error, the component of measurement error that remains constant in replicate measurements.

The bias can be expressed in absolute measurement units or as a percentage relative to the known value. A point estimate is a single value that is the best estimate of the true unknown parameter; a confidence interval is a range of values and indicates the uncertainty of the estimate.

The bias is an estimate of the true unknown bias in a single study. If the study were repeated, the estimate would be expected to vary from study to study. Therefore, if a single estimate is compared directly to 0 or compared to the allowable bias the statement is only applicable to the single study. To make inferences about the true unknown bias you must perform a hypothesis test:

There are two common hypotheses of interest that can be tested:

- Equality test
The null hypothesis states that the bias is equal to 0, against the alternative hypothesis that it is not equal zero. When the test p-value is small, you can reject the null hypothesis and conclude that the bias is different to zero.

It is important to remember that a statistically significant p-value tells you nothing about the practical importance of what was observed. For a large sample, the bias for a statistically significant hypothesis test may be so small as to be practically useless. Conversely, although there may some evidence of bias, the sample size may be too small for the test to reach statistical significance, and you may miss an opportunity to discover a true meaningful bias. Lack of evidence against the null hypothesis does not mean it has been proven to be true, the belief before you perform the study is that the null hypothesis is true and the purpose is to look for evidence against it. An equality test at the 5% significance level is equivalent to comparing a 95% confidence interval to see if it includes zero.

- Equivalence test
The null hypothesis states that the bias is outside an interval of practical equivalence, against the alternative hypothesis that the bias is within the interval considered practically equivalent. When the test p-value is small, you can reject the null hypothesis and conclude that the bias is practically equivalent, and within the specified interval.

An equivalence test is used to prove a bias requirement can be met. The null hypothesis states the methods are not equivalent and looks for evidence that they are in fact equivalent. An equivalence hypothesis test at the 5% significance level is the same as comparing the 90% confidence interval to the allowable bias interval.

Related tasks

Estimating the bias of a measurement system

Related information

ISO. (1994). ISO 5725: Accuracy (Trueness and Precision) of Measurement Methods and Results. International Organization for Standardization.

Estimating the bias of a measurement system

Estimate the trueness of a measurement system or measurement procedure.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Trueness.
The analysis task pane opens.

3. In the Y drop-down list, select the measured quantity variable.

4. If there are multiple levels of measurement, in the By drop-down list, select the level variable, and then:

- If the values are identifiers, select the Identifier, and then in the Assigned values grid, under the Value column alongside each level, type the value.

- If the values are dilutions made by diluting a high pool or mixing high and low pools, select the Relationship, and then select Mixture, Dilution, or Addition based on how the levels were prepared. In the Assigned values grid, under the Value column for the first and/or last level, type the value (intermediate values are automatically calculated using relative values).

- If the values are known/expected/assigned values, select the Known values.

5. If the reference materials package insert includes an estimate of the uncertainty of the assigned values, select the Uncertainty in assigned values check box, and then in the Assigned values grid, under the SE column for each level, type the value:

- If the estimate is expressed as standard uncertainty or combined standard uncertainty, use the value.

- If the estimate is expressed as a 95% confidence interval (CI) use the value CI/2, or if the confidence interval is expressed as upper and lower limits use the value (CI upper limit- CI lower limit)/2.

- If the estimate is expressed as expanded uncertainty (U) with a coverage factor (% or k) use the value U/k. If coverage is 95%, use the value U/2; if coverage is 99%, use the value U/3.

6. Optional: To compare the bias against performance requirements:

- If the allowable bias is a constant or proportional value across the measuring interval, select Across measuring interval, and then in the Absolute edit box, type the bias in measurement units, and/or in the Relative edit box, type the bias as a percentage (suffix with % symbol).
Note: The allowable bias is the greater of the absolute bias and the relative bias for each level. Therefore, with a absolute bias of 5mg/dL and a relative bias of 10%, the allowable bias will be set at 5mg/dL for all values 0 mg/dL up to 50mg/dL and then at 10% of assigned value for values above 50mg/dL.

- If the allowable bias varies for each level, select Each level and then in the Allowable bias grid, under the Absolute / Relative column alongside each level, type the bias in measurement units, or the bias as a percentage (suffix with % symbol).

Note: To compare if the value of the reference material is equal to the assigned value, see Testing against an assigned value.

7. Click Calculate.

Related concepts

Bias

Related tasks

Testing against an assigned value

Testing bias against allowable bias

Related reference

Dataset layout

Related information

CLSI. (2014b). User Verification of Performance for Precision and Trueness (EP15-A3). Clinical and Laboratory Standards Institute.

Testing against an assigned value

Test if the value of the reference material is equal to the assigned value; that is the bias is zero.

You must have already completed the task:

- Estimating the bias of a measurement system

It is possible for the bias from a study to be different to 0, but for the difference to be due to the random error in your study. The smaller your study, the larger the bias has to be to be declared statistically significant. Also, just because bias is statistically significant does not mean that it is of concern, it may still meet your allowable bias performance requirements.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Bias group, click Test Equality.
The analysis task pane Trueness panel opens.

3. Optional: In the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%) and if required select the Familywise error rate check box.

Note: CLSI EP15 uses a 5% familywise significance level so the overall significance level is a maximum of 5% regardless of the number of comparisons (for example, for 1 level the significance level is 5%, for 2 levels the significance level is 5%/2 = 2.5% for each level, for 3 levels the significance level is 5%/3 = 1.6% for each level).

4. Click Recalculate.

Related tasks

Testing bias against allowable bias

Related information

CLSI. (2014b). User Verification of Performance for Precision and Trueness (EP15-A3). Clinical and Laboratory Standards Institute.

Testing bias against allowable bias

Test if the bias meets performance requirements; that is the bias is less than allowable bias.

You must have already completed the task:

- Estimating the bias of a measurement system

If the bias from a study is less than the allowable bias, it meets the performance requirements, but you cannot make any statements relating to probability. An equivalence test allows you to make statements with a given level of confidence about what you observed.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Bias group, click Test Equivalence.
The analysis task pane Trueness panel opens.

3. Optional: In the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%) and if required select the Familywise error rate check box.

Note: CLSI EP15 uses a 5% familywise significance level so the overall significance level is a maximum of 5% regardless of the number of comparisons (for example, for 1 level the significance level is 5%, for 2 levels the significance level is 5%/2 = 2.5% for each level, for 3 levels the significance level is 5%/3 = 1.6% for each level).

4. Enter the performance requirements:

- If the allowable bias is a constant or proportional value across the measuring interval, select Across measuring interval, and then in the Absolute edit box, type the bias in measurement units, and/or in the Relative edit box, type the bias as a percentage (suffix with % symbol).
Note: The allowable bias is the greater of the absolute bias and the relative bias for each level. Therefore, with a absolute bias of 5mg/dL and a relative bias of 10%, the allowable bias will be set at 5mg/dL for all values 0 mg/dL up to 50mg/dL and then at 10% of assigned value for values above 50mg/dL.

- If the allowable bias varies for each level, select Each level and then in the Allowable bias grid, under the Absolute / Relative column, alongside each level, type the absolute bias in measurement units, or the relative bias as a percentage (suffix with % symbol).

5. Click Recalculate.

Related tasks

Testing against an assigned value

Linearity

Nonlinear bias is a component of bias that cannot be represented by a linear relationship between the measured and true values.

A measurement procedure is linear when there is a mathematically verified straight-line relationship between the measured and true values. It is an important parameter as it allows linear interpolation of results between points.

A linear (straight-line) fit describes a relationship where the measuring system is linear. A polynomial fit describes a relationship where the measuring system is nonlinear. In evaluating linearity, a nonlinear polynomial fit is compared against a linear fit. Typically the 2nd and 3rd order polynomial fits are of interest:

- 2nd-order polynomial fit (curved line), where there is nonlinearity due to increasing or decreasing recovery at higher levels.

- 3rd-order polynomial fit (slanted S-shaped curve), where there is nonlinearity at the ends of the measuring interval due to changing recovery across levels.

Bias due to nonlinearity is measured as the difference between the linear fit and the best fitting polynomial fit.

Related tasks

Estimating the linearity of a measurement procedure (EP06-A)

Related information

Kroll, M. H., & Emancipator, K. (1993). A theoretical evaluation of linearity. Clinical chemistry, 39(3), 405-413.

Emancipator, K., & Kroll, M. H. (1993). A quantitative measure of nonlinearity. Clinical chemistry, 39(5), 766-772.

Hsieh, E., & Liu, J. P. (2008). On statistical evaluation of the linearity in assay validation. Journal of Biopharmaceutical Statistics, 18(4), 677-690.

Hsieh, E., Hsiao, C. F., & Liu, J. P. (2009). Statistical methods for evaluating the linearity in assay validation. Journal of Chemometrics, 23(1), 56-63.

Estimating the linearity of a measurement procedure (EP06-A)

Determine whether a measurement system or procedure provides measured quantity values (within a measuring interval) that are directly proportional to the true value.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Linearity.

3. In the Y drop-down list, select the measured variable.

4. In the By drop-down list, select the level variable, and then:

- If the values are identifiers, select the Identifier, and then in the Assigned values grid, under the Value column alongside each level, type the value.

- If the values are dilutions made by diluting a high pool or mixing high and low pools, select the Relationship, and then select Mixture, Dilution, or Addition based on how the levels were prepared. In the Assigned values grid, under the Value column for the first and/or last level, type the value (intermediate values are automatically calculated using relative values).

- If the values are known/expected/assigned values, select the Known values.

Note: Computation of linearity only requires the relationship between the levels, you do not need to enter the assigned values if they are unknown.

5. Optional: To compare the nonlinearity bias against performance requirements:

- If the allowable nonlinearity bias is a constant or proportional value across the measuring interval, select Across measuring interval, and then in the Absolute edit box, type the bias in measurement units, and/or in the Relative edit box, type the bias as a percentage (suffix with % symbol).
Note: The allowable bias is the greater of the absolute bias and the relative bias for each level. Therefore, with a absolute bias of 5mg/dL and a relative bias of 10%, the allowable bias will be set at 5mg/dL for all values 0 mg/dL up to 50mg/dL and then at 10% of assigned value for values above 50mg/dL.

- If the allowable nonlinearity bias varies for each level, select Each level and then in the Allowable nonlinearity grid, under the Absolute / Relative column alongside each level, type the bias in measurement units, or the bias as a percentage (suffix with % symbol).

6. Optional: On the Fit Model panel, in the Fit drop-down list, select:

Option

Description

Best significant term polynomial (CLSI EP6-A)

Fits a 2nd- and 3rd-order polynomial fit, then determines if either are better than the linear fit by testing whether the nonlinear terms (polynomial fit coefficients) are statistically significant. The 3rd-order polynomial is used if it has significant nonlinear terms, the 2nd-order polynomial is used if it has significant nonlinear terms, otherwise the measurement procedure is assumed to be linear. Recommended by CLSI EP6.

Forward stepwise polynomial LoF

Fits a linear fit and performs a lack-of-fit (LoF) significance test to determine how well the model fits the data. If the fit is significantly worse than expected, a 2nd-order polynomial is fit and the LoF test is repeated. If the 2nd-order polynomial fit is not significantly worse, it is used, otherwise a 3rd-order polynomial is fit. Recommended by Liu and others.

Polynomial

Fits a polynomial model of given order.

Note: If the precision of the measurement procedure is poor nonlinearity can be difficult to detect as neither of the polynomial fits will be significantly better than the linear fit, due to the amount of random error in the measurements.

7. Click Calculate.

Related concepts

Linearity

Related reference

Dataset layout

Related information

CLSI. (2003). Evaluation of the Linearity of Quantitative Measurement Procedures (EP6-A). Clinical and Laboratory Standards Institute.

Estimating the linearity of a measurement procedure (EP06-Ed2)

Determine whether a measurement system or procedure provides measured quantity values (within a measuring interval) that are directly proportional.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Linearity.

3. In the Y drop-down list, select the measured variable.

4. In the By drop-down list, select the level variable, and then:

- If the values are identifiers, select the Identifier, and then in the Assigned values grid, under the Value column alongside each level, type the value.

- If the values are dilutions made by diluting a high pool or mixing high and low pools, select the Relationship, and then select Mixture, Dilution, or Addition based on how the levels were prepared. In the Assigned values grid, under the Value column for the first and/or last level, type the value (intermediate values are automatically calculated using relative values).

- If the values are known/expected/assigned values, select the Known values.

Note: Computation of linearity only requires the relationship between the levels, you do not need to enter the assigned values if they are unknown.

5. Optional: To compare the nonlinearity bias against performance requirements:

- If the allowable nonlinearity bias is a constant or proportional value across the measuring interval, select Across measuring interval, and then in the Absolute edit box, type the bias in measurement units, and/or in the Relative edit box, type the bias as a percentage (suffix with % symbol).
Note: The allowable bias is the greater of the absolute bias and the relative bias for each level. Therefore, with a absolute bias of 5mg/dL and a relative bias of 10%, the allowable bias will be set at 5mg/dL for all values 0 mg/dL up to 50mg/dL and then at 10% of assigned value for values above 50mg/dL.

- If the allowable nonlinearity bias varies for each level, select Each level and then in the Allowable nonlinearity grid, under the Absolute / Relative column alongside each level, type the bias in measurement units, or the bias as a percentage (suffix with % symbol).

6. On the Fit Model panel, in the Fit drop-down list, select Linear.

7. In the X drop-down list, select Expected values and in the Y down list, select Mean.

8. In the Weights drop down list, select

Option

Description

None

Fit an ordinary regression. Use when measurement procedure exhibits constant SD over the measuring interval.

Var(Y level)

Fit a weighted regression with weight based on the variance at each level. Recommended when number of replicates per level is 4 or more.

Var(Y level pooled)

Fit a weighted regression with weight based on the pooled variance over a subinterval of levels. Recommended when number of replicates per level is 2 or 3.

VarFn(Y)

Fit a weighted regression with weight based on the variance function at the mean of each level. Recommended when the precision can be modelled by a variance function.

9. If the levels are made by a dilution of a high level, select Force through zero check box. Otherwise, if the levels are produced by mixing a high and low level, clear the Force though origin check box.

10. Click Calculate.

Related reference

Dataset layout

Related information

CLSI. (2020). Evaluation of the Linearity of Quantitative Measurement Procedures (EP6-Ed2). Clinical and Laboratory Standards Institute.

Interferences

Interference bias is a component of bias caused by nonspecificity attributable to the presence of a specific interfering substance.

Detection capability

Detection capability describes the performance of a measurement system at small measured quantity values.

- Limit of blank (LoB)
Limit of blank (also known as the critical value) is the highest quantity value that is likely to be observed, with a stated probability, for a blank material.

- Limit of detection (LoD)
Limit of detection (also known as the minimum detectable value) is the quantity value, for which the probability of falsely claiming the absence of a measurand in a material is beta, given probability (alpha) of falsely claiming its presence.

- Limit of quantitation (LoQ)
Limit of quantitation is the smallest quantity value that meets the requirements for intended use.

Limit of blank (LoB)

Limit of blank (also known as the critical value) is the highest quantity value that is likely to be observed, with a stated probability, for a blank material.

Related concepts

Limit of detection (LoD)

Related tasks

Estimating the detection limit of a measurement system

Limit of detection (LoD)

Limit of detection (also known as the minimum detectable value) is the quantity value, for which the probability of falsely claiming the absence of a measurand in a material is beta, given probability (alpha) of falsely claiming its presence.

Related concepts

Limit of blank (LoB)

Related tasks

Estimating the detection limit of a measurement system

Estimating the detection limit using a precision profile

Estimating the detection limit using a probit fit

Related information

Linnet, K., & Kondratovich, M. (2004). Partly nonparametric approach for determining the limit of detection. Clinical chemistry, 50(4), 732-740.

Currie, L. A. (1999). Detection and quantification limits: origins and historical overview. Analytica Chimica Acta, 391(2), 127-134.

Sadler, W. A., (2016). Using the variance function to estimate limit of blank, limit of detection and their confidence intervals. Ann Clin Biochem, 53, 141-149.

Estimating the detection limit of a measurement system

Estimate the limit of detection of a measurement system or procedure.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Precision and then click Detection Capability.
The analysis task pane opens.

3. In the Y drop-down list, select the measured variable.

4. In the By drop-down list, select the level variable, and then:

- If the values are identifiers, select the Identifier, and then in the Assigned values grid, under the Value column alongside each level, type the value.

- If the values are dilutions made by diluting a high pool or mixing high and low pools, select the Relationship, and then select Mixture, Dilution, or Addition based on how the levels were prepared. In the Assigned values grid, under the Value column for the first and/or last level, type the value (intermediate values are automatically calculated using relative values).

- If the values are known/expected/assigned values, select the Known values.

Note: Computation of detection limits only requires the assigned value of 0 for blank materials. You do not need to assign values to other materials.

5. In the LoB Estimator drop-down list, select:

Option

Description

Normal quantile

Estimate the LoB using the 5% upper of the distribution of blank values (levels with assigned value 0).
Use when values are normally distributed and not truncated at 0.

Quantile

Estimate the LoB using the 95th percentile of the distribution of blank values.
Use when values are truncated at zero.

Known value

Use specific value as the LoB.

6. In the Alpha edit box, type the probability that a blank material gives a result greater than the critical value (LoB), when in fact the substance is not present (typically 5%).

7. In the Beta edit box, type the probability that a non-blank material gives a result less than the critical value (LoB) when in fact the substance is present (typically 5%).

8. Click Calculate.

Related concepts

Limit of detection (LoD)

Limit of blank (LoB)

Related reference

Dataset layout

Related information

CLSI. (2012). Evaluation of Detection Capability for Clinical Laboratory Measurement Procedures (EP17-A2). Clinical and Laboratory Standards Institute.

Estimating the detection limit using a precision profile

Estimate the limit of detection of a measurement system or procedure using a precision profile variance function.

You must have already completed the task:

- Fitting a variance function

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the MSA group, click Detection Capability.
The analysis task pane Detection Capability panel opens.

3. In the LoB Estimator drop-down list, select:

Option

Description

Normal quantile

Estimate the LoB using the 5% upper of the distribution of blank values (levels with assigned value 0).
Use when values are normally distributed and not truncated at 0.

Quantile

Estimate the LoB using the 95th percentile of the distribution of blank values.
Use when values are truncated at zero.

Known value

Use specific value as the LoB.

4. If you selected Normal quantile, in the SD drop-down list, select:

Option

Description

Blank material

Estimate based on material assigned a value of 0.

Precision profile at blank material value

Estimate based on the precision profile variance function at the mean value of the material assigned a value of 0.

Precision profile at zero

Estimate based on the precision profile variance function at the value 0.
Use when it is not possible to obtain blank material, but the profile function is representative of values near zero.

5. In the Alpha edit box, type the probability that a blank material gives a result greater than the critical value (LoB), when in fact the substance is not present (typically 5%).

6. In the SD drop-down list, select Precision profile function.

7. In the Beta edit box, type the probability that a non-blank material gives a result less than the critical value (LoB) when in fact the substance is present (typically 5%).

8. Click Calculate.

Related concepts

Limit of detection (LoD)

Limit of blank (LoB)

Variance function

Related reference

Dataset layout

Related information

CLSI. (2012). Evaluation of Detection Capability for Clinical Laboratory Measurement Procedures (EP17-A2). Clinical and Laboratory Standards Institute.

Estimating the detection limit using a probit fit

Fit a probit regression model to estimate the limit of detection.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Fit Model, and then click Probit.
The analysis task pane opens.

3. In the Y drop-down list, select the binary response variable.

4. In the Event drop-down list, select the positive outcome.

5. In the X drop-down list, select the level variable.

6. Next to the X drop-down list, click the drop-down menu, and then in the Transform drop-down list, select Log 10

7. In the Plot drop-down list, select Scaled - X against Y.

8. Clear the Individual check box, and select the Aggregate check box.

9. On the Analyse-it ribbon tab, in the Fit Model group, click Predict, and then click X given Probability.

10. On the analysis task pane, in the Fit panel, in the Predict X given Probability edit box, type 0.95

11. Click Calculate.

On the analysis report, in the Predict X for Given Probability section, the predicted level at 0.95 probability is the limit of detection.

Related concepts

Limit of detection (LoD)

Limit of quantitation (LoQ)

Limit of quantitation is the smallest quantity value that meets the requirements for intended use.

There is no set procedure to determine the limit of quantitation. Some authors suggest setting separate goals for bias and imprecision. Others such as Westgard use a model to combine the bias and imprecision into "total error" and compare it to a total allowable error goal. Others prefer to avoid the use of models and estimate total error directly using difference between the results of a method and a reference method (Krouwer, 2002). In other areas such as immunoassays the limit of quantitation is often defined as functional sensitivity where the precision profile function CV is equal 20%.

Related information

Krouwer, J. S. (2002). Setting performance goals and evaluating total analytical error for diagnostic assays. Clinical Chemistry, 48(6), 919-927.

Study design

Measurement systems analysis study requirements and dataset layout.

Requirements

- A quantitative variable.

- 1 or more optional factor variables indicating the random effects of interest.

- At least 2 replicates at each level.

Dataset layout

Use a column for the measured variable (Measured value) and optionally a by variable (Level); each row is a separate measurement.

Level (optional)

Measured value

120

121

120

118

120

124

120

120

120

116

…

…

240

240

240

246

240

232

240

241

240

240

…

…

Dataset layout for 1 random factor

Use a column for the measured variable (Measured value), and 1 columns for the random factor (Run), and optionally a by variable (Level); each row is a separate measurement.

Level (optional)

Run

Measured value

120

1

121

120

1

118

120

1

…

120

2

120

120

2

116

120

2

…

120

3

…

120

…

…

…

…

…

240

1

240

240

1

242

240

1

…

240

2

260

240

2

238

240

2

…

240

3

…

240

…

…

…

…

…

Dataset layout for 2 random factors

Use a column for the measured variable (Measured value), and 2 columns for the random factors (Day, Run), and optionally a by variable (Level); each row is a separate measurement.

Level (optional)

Day

Run

Measured value

120

1

1

121

120

1

1

118

120

1

1

…

120

1

2

120

120

1

2

116

120

1

2

…

120

1

3

…

120

1

…

…

120

2

1

124

120

2

1

119

120

2

1

…

120

2

2

118

120

2

2

121

120

2

2

…

120

2

3

…

120

2

…

…

…

…

…

Alternate dataset layout

Use multiple columns for the replicates of the measured variable (Measured value) and a by variable (Level), each row is a separate combination of level and factors.

Level

Measured value

1

0.7

0.9

…

2

4.6

4.1

…

3

6.5

6.9

…

4

11

12.2

…

Note: All the above dataset layouts can arrange replicate measurements in a single row rather than in multiple rows.

Chapter 20. Reference interval

A reference interval (sometimes called a reference range or normal range) describes the range of values of a measured quantity in healthy individuals.

- Reference limits
A reference limit defines a value where a given proportion of reference values are less than or equal to. A reference interval defines the interval between a lower and upper reference limit that includes a given proportion of the reference values.

- Histogram
A histogram shows the distribution of the data to assess the central tendency, variability, and shape.

- Box plot
A box plot shows the five-number summary of the data – the minimum, first quartile, median, third quartile, and maximum. An outlier box plot is a variation of the skeletal box plot that also identifies possible outliers.

- Normal probability (Q-Q) plot
A normal probability plot, or more specifically a quantile-quantile (Q-Q) plot, shows the distribution of the data against the expected normal distribution.

Reference limits

A reference limit defines a value where a given proportion of reference values are less than or equal to. A reference interval defines the interval between a lower and upper reference limit that includes a given proportion of the reference values.

The process of defining a reference interval is that reference individuals compromise a reference population from which is selected a reference sample group on which are determined reference values on which is observed a reference distribution from which are calculated reference limits that define a reference interval.

A point estimate of a reference limit is a single value that is the best estimate of the true unknown parameter; a confidence interval is a range of values and indicates the uncertainty of the estimate. It is important to remember that the width of the confidence interval is dependent upon sample size and that large sample sizes are required to produce narrow confidence intervals, particularly for skewed distributions (Linnet, 2000).

There are numerous quantile estimators that may define the reference limits. The best estimator depends heavily on the shape of the distribution.

- Normal theory quantile
A quantile estimator derived from a normally distributed population with unknown mean and standard deviation.

- Quantile
A distribution-free (non-parametric) quantile estimator based on the order statistics (the sorted values in the sample).

- Bootstrap quantile
A distribution-free (non-parametric) quantile estimator that is the median of a set of quantiles calculated by re-sampling the original sample a large number of times and computing a quantile for each sample.

- Harrell-Davis quantile
A distribution-free (non-parametric) estimator that is a weighted linear combination of order statistics. It is substantially more efficient than the traditional estimator based on one or two order statistics.

- Robust bi-weight quantile
A quantile estimator that uses robust estimators of location and spread, which resist the effects of extreme observations.

Related tasks

Estimating a reference limit or interval

Transferring reference limits

Related information

Linnet, K. (2000). Nonparametric estimation of reference intervals by simple and bootstrap-based procedures. Clinical Chemistry, 46(6), 867-869.

Normal theory quantile

A quantile estimator derived from a normally distributed population with unknown mean and standard deviation.

A normal theory quantile is most powerful when a random sample is from a population with a normal distribution.

Estimator

Description

MVUE

The uniformly minimum unbiased variance quantile estimator uses the sample mean and unbiased standard deviation as the best estimate of the population parameters (Xbar ± Z(alpha) * (s / c4(n))). The factor c4(n) is applied to the sample standard deviation to account for the bias in the estimate for small sample sizes.

t-based

The t-based prediction interval quantile estimator uses the Student's t distribution for a prediction interval (Xbar ± t(alpha, n-1) * s * sqrt(1+1/n)) for a single future observation (Horn, 2005). Note that this method produces a wider interval than the MVUE estimator for small samples.

In many cases, the data are skewed to the right and do not follow a normal distribution. A Box-Cox (or logarithmic) transform can correct the skewness, allowing you to use the Normal theory quantile. If not, a distribution-free estimator may be more powerful (IFCC, 1987).

Related information

Horn, P. S., & Pesce, A. J. (2005). Reference intervals: a user's guide. American Association for Clinical Chemistry.

Solberg, H. E. (1987). Approved recommendation on the theory of reference values. Part 5. Statistical treatment of collected reference values. Determination of reference limits. Clinica Chimica Acta, 170(2), S13-S32.

Quantile

A distribution-free (non-parametric) quantile estimator based on the order statistics (the sorted values in the sample).

There are several definitions for the quantile estimator useful in defining reference limits.

Formula (N is the sample size and p is the quantile)

Description

Np+1/2

Piecewise linear function where the knots are the values midway through the steps of the empirical distribution function. Recommended by Linnet (2000) as having the lowest root-mean-squared-error .

(N+1)p

Linear interpolation of the expectations for the order statistics of the uniform distribution on [0,1]. Recommended by IFCC (1987) and CLSI (2010).

(N+1/3)p+1/3

Linear interpolation of the approximate medians for the order statistics. Has many excellent properties and is approximately median-unbiased regardless of the distribution.

A 95% reference interval (0.025 and 0.975 quantiles) requires a minimum sample size of 39. A 90% confidence interval for a 95% reference interval requires a minimum sample size of 119.

Related information

Linnet, K. (2000). Nonparametric estimation of reference intervals by simple and bootstrap-based procedures. Clinical Chemistry, 46(6), 867-869.

Solberg, H. E. (1987). Approved recommendation on the theory of reference values. Part 5. Statistical treatment of collected reference values. Determination of reference limits. Clinica Chimica Acta, 170(2), S13-S32.

CLSI. (2010). Defining, Establishing, and Verifying Reference Intervals in the Clinical Laboratory (EP28-A3C, formerly C28-A3C). Clinical and Laboratory Standards Institute.

Bootstrap quantile

A distribution-free (non-parametric) quantile estimator that is the median of a set of quantiles calculated by re-sampling the original sample a large number of times and computing a quantile for each sample.

The bootstrap quantile (Linnet, 2000)as providing the lowest root-mean-squared-error (an estimate of the bias and precision in the estimate) for both normal and skewed distributions. Another advantage is that confidence intervals can be computed for smaller sample sizes, although Linnet still recommends a sample size of at least 100.

Related information

Linnet, K. (2000). Nonparametric estimation of reference intervals by simple and bootstrap-based procedures. Clinical Chemistry, 46(6), 867-869.

Harrell-Davis quantile

A distribution-free (non-parametric) estimator that is a weighted linear combination of order statistics. It is substantially more efficient than the traditional estimator based on one or two order statistics.

Related information

Harrell, F. E., & Davis, C. E. (1982). A new distribution-free quantile estimator. Biometrika, 69(3), 635-640.

Robust bi-weight quantile

A quantile estimator that uses robust estimators of location and spread, which resist the effects of extreme observations.

The robust bi-weight estimator (Horn & Pesce 1998) uses robust estimators of location and spread, which resist the effects of extreme observations. It requires a symmetric distribution. If the distribution is skewed, you should apply a transform to achieve symmetry, or create a new sample by mirroring the sample around the median to form a symmetric distribution (Horn, 1990), before applying the bi-weight estimator.

Related information

Horn, P. S., & Pesce, A. J. (2005). Reference intervals: a user's guide. American Association for Clinical Chemistry.

Horn, P. S. (1990). Robust quantile estimators for skewed populations. Biometrika, 77(3), 631-636.

Horn, P. S., Pesce, A. J., & Copeland, B. E. (1998). A robust approach to reference interval estimation and evaluation. Clinical Chemistry, 44(3), 622-631.

Horn, P. S., Pesce, A. J., & Copeland, B. E. (1999). Reference interval computation using robust vs. parametric and nonparametric analyses. Clinical Chemistry, 45(12), 2284-2285.

Estimating a reference limit or interval

Estimate the reference limit or reference interval using a quantile estimator.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Diagnostic, and then under the Reference Interval heading, click the estimator.
The analysis task pane opens.

3. In the Y drop-down list, select the variable.

4. In the Type drop-down list, select:

Option

Description

Reference interval

Estimate lower and upper reference limits that a stated proportion of values are between.

Reference limit

Estimate a single reference limit that a stated proportion of values are less than.

5. In the Proportion inside interval/Proportion less than edit box, type the reference level as a percentage.

6. In the Method drop-down list, select the computation method.

7. In the Confidence interval edit box, type the confidence level as a percentage, or type - to suppress the confidence interval, and then in the drop-down list, select the confidence bounds.

8. Optional: To split the reference values by factors such as gender or age:

1. On the Analyse-it ribbon tab, in the Reference Interval group, click Partition

2. In the Factors list, select the factor variables.

3. In the Partitions list, select a partition.
The tasks change to the analysis options for that partition.

4. Optional: By default, the analysis options are the same for each partition, set the specific analysis options for the partition as required.

5. Repeat steps 8.c through 8.d for each partition.

9. Click Calculate.

Related concepts

Reference limits

Related tasks

Transforming the reference distribution

Testing the normality of a reference distribution

Related reference

Dataset layout

Histogram

A histogram shows the distribution of the data to assess the central tendency, variability, and shape.

A histogram for a quantitative variable divides the range of the values into discrete classes, and then counts the number of observations falling into each class interval. The area of each bar in the histogram is proportional to the frequency in the class. When the class widths are equal, the height of the bar is also proportional to the frequency in the class.

Choosing the number of classes to use can be difficult as there is no "best," and different class widths can reveal or hide features of the data. Scott's and Freedman-Diaconis' rules provide a default starting point, though sometimes particular class intervals make sense for a particular problem.

The histogram reveals if the distribution of the data is normal, skewed (shifted to the left or right), bimodal (has more than one peak) and so on. Skewed data can sometimes be transformed to normal using a transformation. Bi-modality often indicates that there is more than one underlying population in the data. Individual bars distanced from the bulk of the data can indicate the presence of an outlier.

Box plot

A box plot shows the five-number summary of the data – the minimum, first quartile, median, third quartile, and maximum. An outlier box plot is a variation of the skeletal box plot that also identifies possible outliers.

An outlier box plot is a variation of the skeletal box plot, but instead of extending to the minimum and maximum, the whiskers extend to the furthest observation within 1.5 x IQR from the quartiles. Possible near outliers (orange plus symbol) are identified as observations further than 1.5 x IQR from the quartiles, and possible far outliers (red asterisk symbol) as observations further than 3.0 x IQR from the quartiles. You should investigate each possible outlier before deciding whether to exclude it, as even in normally distributed data, an outlier box plot identifies approximately 0.7% of observations as possible outliers.

Normal probability (Q-Q) plot

A normal probability plot, or more specifically a quantile-quantile (Q-Q) plot, shows the distribution of the data against the expected normal distribution.

For normally distributed data, observations should lie approximately on a straight line. If the data is non-normal, the points form a curve that deviates markedly from a straight line. Possible outliers are points at the ends of the line, distanced from the bulk of the observations.

Testing the normality of a reference distribution

Test if a reference distribution is normal.

You must have already completed the task:

- Estimating a reference limit or interval

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Reference Interval group, click Normality.
The analysis task pane Normality panel opens with the Normal plot check box selected.

3. Optional: To perform a formal normality hypothesis test, select the Normality test check box.

4. Click Recalculate.

Related concepts

Normal probability (Q-Q) plot

Normality hypothesis test

Transforming the reference distribution

Transform the reference values to achieve a normal distribution.

You must have already completed the task:

- Estimating a reference limit or interval

You should transform the data when the distribution does not match the assumptions of the quantile estimator. If you cannot transform the distribution to meet the assumptions of the estimator you should consider using another estimator.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Reference interval group, click Transform, and then click the transformation.
The analysis task pane Transform panel opens.

3. Set the transform parameters:

- If the transform is the logarithm, in the Base edit box, type the base of the logarithm or enter - for the natural logarithm.

- If the transform is Box-Cox, in Shift and Lambda edit boxes, type the parameters, or to estimate them from the data, click Estimate unknown parameter.

- If the transform is exponential, in the Gamma edit box, type parameter.

- If the transform is 2-stage exponential/modulus, in the Gamma and Kappa edit boxes, type parameters.

4. Click Recalculate.

Transferring reference limits

Transfer a reference limit or reference interval from one method or laboratory to another.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Diagnostic, and then under the Reference Interval heading, click Transference.
The analysis task pane opens.

3. In the Y drop-down list, select the variable.

4. Optional: To split the reference values by factors such as gender or age:

1. On the Analyse-it ribbon tab, in the Reference Interval group, click Partition

2. In the Factors list, select the factor variables.

3. In the Partitions list, select a partition.
The tasks change to the analysis options for that partition.

4. Optional: By default, the analysis options are the same for each partition, set the specific analysis options for the partition as required.

5. Repeat steps 4.c through 4.d for each partition.

5. In the Reference level and Reference interval edit boxes, type the reference level as a percentage and the upper/lower reference limits.

6. Optional: If transferring from another measurement procedure, under the Linear transformation heading, in the Constant and Proportional edit boxes, type the constant and proportional average bias (available from the method comparison study) to translate the old reference limits to the new measurement procedure.

7. Select the Hypothesis test check box.

8. In the Hypotheses drop-down list, select the null and alternative hypotheses.

The default null hypothesis is that the proportion of observations in the reference interval is equal to the reference level (95% default) against the alternative hypothesis that there are more or fewer observations in the interval. CLSI EP28-A3 performs a one-sided hypothesis test that proportion inside the interval is greater than or equal to 95% against the alternative that less than 95% are inside the interval.

9. In the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

You need at least 20 samples for a one-sided hypothesis test at the 5% significance level. More samples are required to be able to detect a significant difference for the two-sided hypothesis test, or when a smaller significance level is required.

10. If you selected partition factors, in the Partitions list, select another partition, and then repeat steps 5 through 9 for all partitions.

11. Click Calculate.

Related concepts

Reference limits

Related reference

Dataset layout

Related information

CLSI. (2010). Defining, Establishing, and Verifying Reference Intervals in the Clinical Laboratory (EP28-A3C, formerly C28-A3C). Clinical and Laboratory Standards Institute.

Study design

Reference interval study requirements and dataset layout.

Requirements

- A quantitative variable of the reference values.

- 1 or more factor variables.

Dataset layout

Use a column for the variable (Reference value), and optionally additional columns for each partition factor (Sex); each row has the values of the variables for a case (Subject).

Subject (optional)

Sex (optional)

Reference value

1

Male

121

2

Male

118

3

Female

124

4

Female

120

5

Male

116

6

Male

…

7

Female

100

8

Male

115

9

Female

102

10

Female

98

11

Male

118

…

…

…

Chapter 21. Diagnostic performance

Diagnostic performance evaluates the ability of a qualitative or quantitative test to discriminate between two subclasses of subjects.

- Measures of diagnostic accuracy
Diagnostic accuracy measures the ability of a test to detect a condition when it is present and detect the absence of a condition when it is absent.

- ROC plot
ROC (receiver operating characteristic) curves show the ability of a quantitative diagnostic test to classify subjects correctly as the decision threshold is varied.

- Area under the curve (AUC)
The area under (a ROC) curve is a measure of the accuracy of a quantitative diagnostic test.

- Difference between the areas under two curves
The difference between areas under the ROC curves compares the accuracy of two or more diagnostic tests.

- Decision thresholds
A decision threshold is a value that dichotomizes the result of a quantitative test to a simple binary decision.

- Decision plot
A decision plot shows a measure of performance (such as sensitivity, specificity, likelihood ratios, or predictive values) against all decision thresholds to help identify optimal decision threshold.

Measures of diagnostic accuracy

Diagnostic accuracy measures the ability of a test to detect a condition when it is present and detect the absence of a condition when it is absent.

Comparison of the result of a diagnostic test to the true known condition of each subject classifies each outcome as:

True positive (TP)

Test result correctly identifies the presence of the condition.

False positive (FP)

Test result incorrectly identifies the presence of the condition when it was absent.

True negative (TN)

Test result correctly identifies the absence of the condition.

False negative (FN)

Test result incorrectly identifies the absence of the condition when it was present.

A perfect diagnostic test can discriminate all subjects with and without the condition and results in no false positive or false negatives. However, this is rarely achievable, as misdiagnosis of some subjects is inevitable. Measures of diagnostic accuracy quantify the discriminative ability of a test.

- Sensitivity / Specificity
Sensitivity and specificity are the probability of a correct test result in subjects with and without a condition, respectively.

- Likelihood ratios
Likelihood ratios are the ratio of the probability of a specific test result for subjects with the condition against the probability of the same test result for subjects without the condition.

- Predictive values
Predictive values are the probability of correctly identifying a subject's condition given the test result.

- Youden J
Youden's J is the likelihood of a positive test result in subjects with the condition versus those without the condition. It is also the probability of an informed decision (as opposed to a random guess).

Sensitivity / Specificity

Sensitivity and specificity are the probability of a correct test result in subjects with and without a condition, respectively.

Sensitivity (true positive fraction, TPF) measures the ability of a test to detect the condition when it is present. It is the probability that the test result is positive when the condition is present.

Specificity (true negative fraction, TNF) measures the ability of a test to detect the absence of the condition when it is not present. It is the probability that the test result is negative when the condition is absent.

Related concepts

Measures of diagnostic accuracy

Related tasks

Estimating sensitivity and specificity of a diagnostic test

Comparing the sensitivity and specificity two diagnostic tests

Likelihood ratios

Likelihood ratios are the ratio of the probability of a specific test result for subjects with the condition against the probability of the same test result for subjects without the condition.

A likelihood ratio of 1 indicates that the test result is equally likely in subjects with and without the condition. A ratio > 1 indicates that the test result is more likely in subjects with the condition than without the condition, and conversely, a ratio < 1 indicates that the test result is more likely in subjects without the condition. The larger the ratio, the more likely the test result is in subjects with the condition than without; likewise, the smaller the ratio, the more likely the test result is in subjects without than with the condition.

The likelihood ratio of a positive test result is the ratio of the probability of a positive test result in a subject with the condition (true positive fraction) against the probability of a positive test result in a subject without the condition (false positive fraction). The likelihood ratio of a negative test result is the ratio of the probability of a negative test result in a subject with the condition (false negative fraction) to the probability of a negative test result in a subject without the condition (true negative fraction).

Related concepts

Measures of diagnostic accuracy

Predictive values

Predictive values are the probability of correctly identifying a subject's condition given the test result.

Predictive values use Bayes' theorem along with a pre-test prior probability (such as the prevalence of the condition in the population) and the sensitivity and specificity of the test to compute the post-test probability (predictive value).

The positive predictive value is the probability that a subject has the condition given a positive test result; the negative predictive value is the probability that a subject does not have the condition given a negative test result.

Related concepts

Measures of diagnostic accuracy

Youden J

Youden's J is the likelihood of a positive test result in subjects with the condition versus those without the condition. It is also the probability of an informed decision (as opposed to a random guess).

Youden's J index combines sensitivity and specificity into a single measure (Sensitivity + Specificity - 1) and has a value between 0 and 1. In a perfect test, Youden's index equals 1. It is also equivalent to the vertical distance above the diagonal no discrimination (chance) line to the ROC curve for a single decision threshold.

Related concepts

Measures of diagnostic accuracy

Estimating sensitivity and specificity of a diagnostic test

Estimate the sensitivity/specificity and other measures of the accuracy of a binary diagnostic test (or a quantitative test at a specific decision threshold).

You can use this task with up to two binary tests or quantitative tests at a specified decision threshold. To show the sensitivity/specificity for all possible thresholds of a quantitative test you should use ROC curves.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Diagnostic, and then click Binary (Sensitivity / Specificity).
The analysis task pane opens.

3. In the True state drop-down list, select the true condition variable.

4. In the Positive event drop-down list, select the state that indicates the presence of the condition/event of interest.

5. In the Y drop-down list(s), select the test variable(s).

6. If the Y variable is binary, in the Positive event drop-down list, select the state that indicates a positive test.

7. If the Y variable is quantitative, click the menu next to the variable and select Cut-off, and then enter the criteria to recode the variable into a binary variable.

8. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

9. Optional: Select the appropriate TP/TN/FP/FN, Sensitivity / Specificity, Likelihood ratio, Predictive value check boxes.

10. Click Calculate.

Related concepts

Sensitivity / Specificity

Related tasks

Comparing the sensitivity and specificity two diagnostic tests

Related reference

Dataset layout

Comparing the sensitivity and specificity two diagnostic tests

Compare the sensitivity and specificity of two diagnostic tests and make inferences about the differences.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Diagnostic, and then click Binary (Sensitivity / Specificity).
The analysis task pane opens.

3. In the Model drop-down menu, select the number of tests and the type of study design.

Option

Description

2 paired tests

Compare 2 tests where the results are on the same subjects.

2 independent tests / groups

Compare 2 tests where the results are on different subjects, or 1 test applied to 2 different groups of subjects.

4. In the True state drop-down list, select the true condition variable.

5. In the Positive event drop-down list, select the state that indicates the presence of the condition/event of interest.

6. If comparing 2 independent tests, in the Y drop-down list, select the diagnostic test variable, and then in the Factor drop-down list, select the factor variable (test or group indicator).

7. If comparing 2 paired tests, in the Y drop-down lists, select the diagnostic test variables.

8. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

9. On the Analyse-it ribbon tab, in the Diagnostic Accuracy group, click Test, and then click:

Option

Description

Equality

Test if the sensitivities/specificities of two tests are equal.

Equivalence

Test if the sensitivities/specificities of two tests are equivalent within a practical difference.

Non-inferiority

Test if the sensitivity/specificity of a new test is not inferior to a standard test.

The default options use a Miettinen-Nurminen or Tango score confidence interval, and a Score Z test. The Newcombe score confidence interval is approximate not based on the same evidence function, and rejection of a hypothesis based on the confidence limits and hypothesis test may give conflicting results.

10. In the analysis task pane, under the hypotheses drop-down list:

- If testing a hypothesis of equivalence, in the Equivalent difference, type the smallest practical difference that would be considered the same.

- If testing a hypothesis of non-inferiority, in the Standard drop-down list, select the standard diagnostic test, and then in the Smallest difference edit box, type the smallest difference that would be considered inferior.

11. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

12. Click Calculate.

Related concepts

Sensitivity / Specificity

Decision thresholds

Decision plot

ROC plot

ROC (receiver operating characteristic) curves show the ability of a quantitative diagnostic test to classify subjects correctly as the decision threshold is varied.

The ROC plot shows sensitivity (true positive fraction) on the horizontal axis against 1-specificity (false positive fraction) on the vertical axis over all possible decision thresholds.

A diagnostic test able to perfectly identify subjects with and without the condition produces a curve that passes through the upper left corner (0, 1) of the plot. A diagnostic test with no ability to discriminate better than chance produces a diagonal line from the origin (0, 0) to the top right corner (1, 1) of the plot. Most tests lie somewhere between these extremes. If a curve lies below the diagonal line (0, 0 to 1, 1), you can invert it by swapping the decision criteria to produce a curve above the line.

An empirical ROC curve is the simplest to construct. Sensitivity and specificity use the empirical distributions for the subjects with and without the condition. This method is nonparametric because no parameters are needed to model the behavior of the curve, and it makes no assumptions about the underlying distribution of the two groups of subjects.

Related concepts

Area under the curve (AUC)

Related tasks

Plotting a single ROC curve

Comparing two or more ROC curves

Related information

Zweig, M. H., & Campbell, G. (1993). Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry, 39(4), 561-577.

Zhou, X. H., Obuchowski, N. A., & McClish, D. K. (2011). Statistical methods in diagnostic medicine. Wiley-Blackwell.

Liu, A., & Bandos, A. I. (2012). Statistical evaluation of diagnostic performance: topics in ROC analysis. CRC Press.

Pepe, M. S. (2003). The statistical evaluation of medical tests for classification and prediction. Oxford University Press.

Plotting a single ROC curve

Plot the receiver-operator characteristic (ROC) curve to visualize the accuracy of a diagnostic test.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Diagnostic, and then under the Accuracy heading, click ROC Curve.
The analysis task pane opens.

3. In the True state drop-down list, select the true condition variable.

4. In the Positive event drop-down list, select the state that indicates the presence of the condition/event of interest.

5. In the Y drop-down list, select the diagnostic test variable.

6. Optional: To show a bi-histogram of the distributions of the diagnostic test by the true condition, On the Analyse-it ribbon tab, in the Diagnostic Accuracy group, click Bi-Histogram

7. Optional: To show the diagnostic accuracy for each decision threshold, On the Analyse-it ribbon tab, in the Diagnostic Accuracy group, click Sensitivity / Specificity, Likelihood ratio, Predictive value.

8. Click Calculate.

Related concepts

ROC plot

Area under the curve (AUC)

Measures of diagnostic accuracy

Related tasks

Testing the area under the curve

Finding the optimal decision threshold

Predicting the decision threshold

Related reference

Dataset layout

Comparing two or more ROC curves

Plot multiple receiver-operator characteristics (ROC) curves to make comparisons between them.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Diagnostic, and then under the Accuracy heading, click:

Option

Description

Compare ROC Curves

Compare 2 or more independent ROC curves.

Compare Correlated ROC Curves

Compare 2 or more paired/correlated ROC curves.

The analysis task pane opens.

3. In the True state drop-down list, select the true condition variable.

4. In the Positive event drop-down list, select the state that indicates the presence of the condition/event of interest.

5. If comparing 2 or more independent ROC curves, in the Y drop-down list, select the diagnostic test variable, and then in the Factor drop-down list, select the grouping variable.

6. If comparing 2 or more paired/correlated ROC curves, in the Y list, select the diagnostic test variables.

7. Click Calculate.

Related concepts

ROC plot

Area under the curve (AUC)

Related tasks

Testing the difference between the areas under two curves

Related reference

Dataset layout

Area under the curve (AUC)

The area under (a ROC) curve is a measure of the accuracy of a quantitative diagnostic test.

A point estimate of the AUC of the empirical ROC curve is the Mann-Whitney U estimator (DeLong et. al., 1988). The confidence interval for AUC indicates the uncertainty of the estimate and uses the Wald Z large sample normal approximation (DeLong et al., 1998). A test with no better accuracy than chance has an AUC of 0.5, a test with perfect accuracy has an AUC of 1.

The interpretation of the AUC is:

- The average value of sensitivity for all possible values of specificity (Zhou, Obuchowski, McClish, 2001).

- The average value of specificity for all possible values of sensitivity (Zhou, Obuchowski, McClish, 2001).

- The probability that a randomly selected subject with the condition has a test result indicating greater suspicion than that of a randomly chosen subject without the condition (Hanley & McNeil, 1982).

AUC can be misleading as it gives equal weight to the full range of sensitivity and specificity values even though a limited range, or specific threshold, may be of practical interest.

Related concepts

ROC plot

Difference between the areas under two curves

Related tasks

Plotting a single ROC curve

Comparing two or more ROC curves

Testing the area under the curve

Related information

Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29-36.

DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 837-845.

Zhou, X. H., Obuchowski, N. A., & McClish, D. K. (2011). Statistical methods in diagnostic medicine. Wiley-Blackwell.

Testing the area under the curve

Test if the area under the curve is better than chance, or some other value.

You must have already completed either of the tasks:

- Plotting a single ROC curve

- Comparing two or more ROC curves

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Diagnostic Accuracy group, click Test, and then click No Discrimination.
The default null hypothesis is that the AUC is less than or equal to 0.5 against the alternative hypothesis that the AUC is greater than the 0.5. This tests if the AUC is better than chance alone.

The analysis task pane ROC Curve panel opens.

3. Optional: In the Hypotheses drop-down list, select the null and alternative hypothesis.

4. Optional: In the Hypothesized value edit box, type the expected value of the parameter under the null hypothesis.

5. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

6. Click Recalculate.

Related concepts

Area under the curve (AUC)

Difference between the areas under two curves

The difference between areas under the ROC curves compares the accuracy of two or more diagnostic tests.

It is imperative when comparing tests that you choose the correct type of analysis dependent on how you collect the data. If the tests are performed on the same subjects (paired design), the test results are usually correlated. Less commonly, you may perform the different tests on different groups of subjects or the same test on different groups of subjects, and the test results are independent. A paired design is more efficient and is preferred whenever possible.

A point estimate of the difference between the area under two curves is a single value that is the best estimate of the true unknown parameter; a confidence interval indicates the uncertainty of the estimate. If the tests are independent, the confidence interval uses the combined variance of the curves and a large sample Wald approximation. If the tests are paired, the standard error incorporates the covariance (DeLong et al., 1998) and uses a large sample Wald approximation.

A hypothesis test for the difference in AUC can test equality, equivalence, or non-inferiority of the diagnostic tests. Inferences about the difference between AUC are made using a Z test. The three hypotheses of interest are:

- Equality
The null hypothesis states that the difference is equal to zero, against the alternative hypothesis that it is not equal to zero. When the test p-value is small, you can reject the null hypothesis and conclude the difference is not equal to zero, the tests are different.

- Equivalence
The null hypothesis states that the difference is less than a lower bound of practical equivalence or greater than an upper bound of practical equivalence, against the alternative hypothesis that the difference is within an interval considered practically equivalent. When the test p-value is small, you can reject the null hypothesis and conclude that the tests are equivalent.

- Non-inferiority
The null hypothesis states that the difference from a standard test is greater than the smallest practical difference against the alternative hypothesis that the difference from the standard test is less than the smallest practical difference. When the test p-value is small, you can reject the null hypothesis and conclude that the test is not inferior to the standard test.

When the ROC curves cross, the difference between the AUC does not provide much useful information.

Related concepts

Area under the curve (AUC)

Related tasks

Testing the difference between the areas under two curves

Related information

DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 837-845.

Hanley, J. A., & McNeil, B. J. (1983). A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology, 148(3), 839-843.

Testing the difference between the areas under two curves

Test if the area under curves is equal, equivalent, or not inferior to another.

You must have already completed the task:

- Comparing two or more ROC curves

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Diagnostic Accuracy group, click Test, and then click:

Option

Description

Equality

Test if the AUCs are equal.

Equivalence

Test if the AUCs are equivalent within a practical difference.

Non-inferiority

Test if the AUC is not inferior to another AUC.

The analysis task pane Comparisons panel opens.

3. In the analysis task pane, under the hypotheses drop-down list:

- If testing a hypothesis of equivalence, in the Contrast drop-down list, select the comparisons, and then in the Equivalent difference, type the smallest practical difference that would be considered the same.

- If testing a hypothesis of non-inferiority, in the Standard drop-down list, select the standard diagnostic test, and then in the Smallest difference edit box, type the smallest difference that would be considered inferior.

4. Optional: To compare the p-value against a predefined significance level, in the Significance level edit box, type the maximum probability of rejecting the null hypothesis when in fact it is true (typically 5% or 1%).

5. Click Recalculate.

Related concepts

Difference between the areas under two curves

Decision thresholds

A decision threshold is a value that dichotomizes the result of a quantitative test to a simple binary decision.

The test result of a quantitative diagnostic test is dichotomized by treating the values above or equal to a threshold as positive, and those below as negative, or vice-versa.

There are many ways to choose a decision threshold for a diagnostic test. For a simple screening test, the decision threshold is often chosen to incur a fixed, true positive, or false positive rate. In more complex cases, the optimal decision threshold depends on both the cost of performing the test and the cost of the consequences of the test result. The costs may include both financial and health-related costs. A simple formula to determine the optimal decision threshold (Zweig & Campbell, 1993) maximizes: Sensitivity - m * (1- Specificity) where m = (Cost FP - Cost TN) / (Cost FN - Cost TP).

Related concepts

Decision plot

Related tasks

Comparing the sensitivity and specificity two diagnostic tests

Finding the optimal decision threshold

Related information

Zweig, M. H., & Campbell, G. (1993). Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry, 39(4), 561-577.

Decision plot

A decision plot shows a measure of performance (such as sensitivity, specificity, likelihood ratios, or predictive values) against all decision thresholds to help identify optimal decision threshold.

Related concepts

Decision thresholds

Related tasks

Comparing the sensitivity and specificity two diagnostic tests

Finding the optimal decision threshold

Finding the optimal decision threshold

Find the optimal decision threshold using information on the relative costs of making correct and incorrect decisions.

You must have already completed either of the tasks:

- Plotting a single ROC curve

- Comparing two or more ROC curves

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Diagnostic Accuracy group, click Find Optimal.
The analysis task pane Decision panel opens.

3. If the sample does not represent the prevalence of the condition in the target population, then, in the Prior probability edit box, type the Prevalence of the condition in the target population.

4. In the TP, FP, TN, FN edit boxes, enter the cost of each type of decision.
The cost depends on the difference in costs between FPs and TNs relative to the difference in costs between FNs and TPs. Therefore, the costs can be expressed in any unit and need not be purely financial costs. The default costs are equal.

5. If you want a plot of the cost for each possible decision threshold:

1. On the Analyse-it ribbon tab, in the Diagnostic Accuracy group, click Decision Threshold.
The analysis task pane Decision panel opens.

2. In the Plot drop-down list, select Cost.

6. Click Recalculate.

Related concepts

Decision thresholds

Decision plot

Related tasks

Comparing the sensitivity and specificity two diagnostic tests

Finding the optimal decision threshold

Predicting the decision threshold

Predict the threshold for a fixed true/false positive rate.

You must have already completed either of the tasks:

- Plotting a single ROC curve

- Comparing two or more ROC curves

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Diagnostic Accuracy group, click Predict, and then click:

Option

Description

Fixed FPF

Predict the TPF (Sensitivity) and Threshold at a fixed FPF

Fixed TPF (Sensitivity)

Predicts the FPF and Threshold at a fixed TPF.

Threshold

Predict the TPF (Sensitivity) and FPF at a decision threshold.

The analysis task pane Decision panel opens.

3. In the TPF / FPF / Threshold edit box, type the value at which to determine the other parameters.

4. If you want a plot of the sensitivity and specificity for each decision threshold:

1. On the Analyse-it ribbon tab, in the Diagnostic Accuracy group, click Decision Threshold.
The analysis task pane Decision panel opens.

2. In the Plot drop-down list, select Sensitivity / Specificity.

5. Click Recalculate.

Related concepts

Decision thresholds

Decision plot

Related tasks

Comparing the sensitivity and specificity two diagnostic tests

Finding the optimal decision threshold

Study design

Diagnostic performance study requirements and dataset layout.

Requirements

- A qualitative or quantitative variable of the measured values or indications (positive/negative) of the diagnostic test.

- A qualitative variable indicating the true state (positive/negative) of each subject.

Dataset layout for a single test

Use a column for the true state variable (State), and a column for the test measured values (Test); each row has the values of the variables for a case (Subject).

Subject (optional)

State

Test

1

Diseased

121

2

Diseased

118

3

Diseased

124

4

Diseased

120

5

Diseased

116

6

…

…

7

Healthy

100

8

Healthy

115

9

Healthy

102

10

Healthy

98

11

Healthy

118

…

…

…

Dataset layout for paired tests

Use a column for the true state variable (State), and a column for each of the test measured values (Test 1, Test 2); each row has the values of the variables for a case (Subject).

Subject (optional)

State

Test 1

Test 2

1

Diseased

121

86

2

Diseased

118

90

3

Diseased

124

91

4

Diseased

120

99

5

Diseased

116

89

6

…

…

…

7

Healthy

100

70

8

Healthy

115

80

9

Healthy

102

79

10

Healthy

98

87

11

Healthy

118

90

…

…

…

Dataset layout for independent tests

Use a column for the true state variable (State), a column for the factor (test or group indicator), and a column for the test measured values (Test); each row has the values of the variables for a case (Subject).

Subject (optional)

State

Test/Group ID

Test result

1

Diseased

1

121

2

Diseased

1

118

3

Diseased

1

124

4

Diseased

1

120

5

Diseased

1

116

6

Healthy

1

100

7

Healthy

1

115

8

Healthy

1

102

9

Healthy

1

98

10

Healthy

1

118

11

…

…

…

12

Diseased

2

86

13

Diseased

2

90

14

Diseased

2

91

15

Diseased

2

99

16

Diseased

2

89

17

Healthy

2

70

18

Healthy

2

80

19

Healthy

2

79

20

Healthy

2

87

21

Healthy

2

90

…

…

…

…

Dataset layout for a single binary test

Use a column for the true state variable (State), and a column for the test indications (Test); each row has the values of the variables for a case (Subject).

Subject (optional)

State

Test

1

Diseased

P

2

Diseased

P

3

Diseased

P

4

Diseased

N

5

Diseased

P

6

Healthy

N

7

Healthy

N

8

Healthy

N

9

Healthy

N

10

Healthy

N

…

…

…

Frequency form dataset layout for a binary test

Use a column for the true state variable (State), and a column for the test indications (Test) and a column for the number of cases (Frequency); each row has the values of the variables and the frequency count.

State

Test

Count

Diseased

Positive

3

Diseased

Negative

1

Healthy

Positive

0

Healthy

Negative

5

Frequency form dataset layout for 2 independent binary tests

Use a column for the true state variable (State), a column for the factor (test or group indicator), a column for the test indications (Test) and a column for the number of cases (Frequency); each row has the values of the variables and the frequency count.

State

Test

Group/Test ID

Count

Diseased

Positive

1

3

Diseased

Negative

1

1

Healthy

Positive

1

0

Healthy

Negative

1

5

Diseased

Positive

2

24

Diseased

Negative

2

2

Healthy

Positive

2

2

Healthy

Negative

2

30

Frequency form dataset layout for 2 paired binary tests

Use a column for the true state variable (State), a column for each of the test indications (Test 1, Test 2) and a column for the number of cases (Frequency); each row has the values of the variables and the frequency count.

State

Test 1

Test 2

Count

Diseased

Positive

Positive

3

Diseased

Negative

Positive

1

Diseased

Positive

Negative

24

Diseased

Negative

Negative

2

Healthy

Positive

Positive

0

Healthy

Negative

Positive

5

Healthy

Positive

Negative

2

Healthy

Negative

Negative

30

Chapter 22. Survival/Reliability

Survival/reliability analyzes the time until an event of interest occurs.

In survival analysis, the variable of interest is the time it takes until an event occurs. Time is the length of time in hours, days, months, weeks, years or some other unit. The event of interest is often death, recovery from surgery, product failure or some other characteristic. A key feature of survival analysis is that the event of interest may not occur during the study's time frame, or contact may be lost with the participant part way through the study. In these cases, the observation is said to be censored, and we only have partial but still important information about the survival time. Most survival data is right-censored because the true survival time has been cut off at the right side of the observed time, giving us an observed survival time shorter than the true survival time.

- Kaplan-Meier survival function
The Kaplan-Meier estimator (also known as the product-limit estimator) is a non-parametric statistic used to estimate the empirical survival function.

- Proportional hazards fit
A semi-parametric model that describes the relationship between a time to event response variable and one or more explanatory variables using a hazard function.

Kaplan-Meier survival function

The Kaplan-Meier estimator (also known as the product-limit estimator) is a non-parametric statistic used to estimate the empirical survival function.

The Kaplan-Meier survival probability at failure time t(f) is the probability of surviving past the previous failure time t(f-1) multiplied by the conditional probability of surviving past time t(f) given survival to at least time t(f).

A plot of the survival probabilities at each ordered failure time produces an empirical survival function (or survival curve) - a step-function starting with a horizontal line at a survival probability of 1 and a step down at each failure time. A censored observation does not produce any step in the survival function, but it is sometimes useful to denote such observations.

A key statistic for describing the survival function are the quartiles (25th, 50th, and 75th quantiles of the survival function). Another statistic is the mean survival time, the area under the survival curve. It is often less useful as many survival functions do not drop to 0 (due to some observations being censored at the end of the study). A workaround, particularly when comparing multiple survival functions, is to restrict the area under the curve to the interval 0, t, which is common to all the survival curves.

Related tasks

Plotting a Kaplan-Meier survival curve

Comparing two or more survival functions

Plotting a Kaplan-Meier survival curve

Plot the Kaplan-Meier survival function to visualize the survival probability over time.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Survival/Reliability, and then click Kaplan-Meier Survival Function.
The analysis task pane opens.

3. In the Y (time to event) drop-down list, select the time until the event of interest variable.

4. Optional: In the Censor drop-down list, select the state variable that indicates whether the time was censored or if the event of interest occured, and then in the Event drop-down list, select the state that indicates the time was censored.

5. Optional: In the X (factor) drop-down list, select the grouping variable.

6. Optional: To show the survival probabilities for each failure time, On the Analyse-it ribbon tab, in the Survival/Reliability group, click S(t).

7. Click Calculate.

Related concepts

Kaplan-Meier survival function

Equality of survival functions test

An equality hypothesis test formally tests if two or more population survival functions are different.

A common goal of survival analysis is to compare the survival function of two or more groups. A formal hypothesis test tests whether two or more survival functions are statistically equal in some overall sense. That is, there is no evidence to suggest that the true (population) survival functions are different. The most popular test is the Log-rank test, although various other tests differ in how much weight they give to the survival probabilities at the start or end of the distribution.

The null hypothesis states that there is no difference between the survival functions, against the alternative hypothesis that at least one of the survival functions is different. When the test p-value is small, you can reject the null hypothesis and conclude the population survival functions differ.

Related tasks

Comparing two or more survival functions

Related reference

Tests for the equality of survival functions

Tests for the equality of survival functions

Tests for the equality of survival functions and their properties and assumptions.

Test

Purpose

Log-rank

Test if the survival functions are equal (uses equal weights).
The most popular test. Equivalent to the Mantel-Haenszel test of the hypothesis that the stratum specific odds-ratio is equal to one.

Wilcoxon

Test if the survival functions are equal with more weight on differences between the survival functions at smaller values of time (higher weights on early death). This test is appropriate when hazard functions vary in ways other than proportionally and when censoring patterns are similar across groups.

Tarone-Ware

Test if the survival functions are equal using weights equal to the sqrt(number at risk) rather than the number at risk used in Wilcoxon. Like the Wilcoxon test, it gives larger weights (although not as large) to earlier failure times. Although less susceptible to the censoring pattern in the data than Wilcoxon’s test, this could remain a problem if large differences in these patterns exist between groups.

Fleming-Harrington

Test if the survival functions are equal using weights kmp multiplied by (1-km)q. When p > q, the test weights earlier failures more than later ones. When p < q, the opposite is true, and more weight is given to later failures than to earlier ones. The Fleming–Harrington test reduces to the log-rank test when p = 1 and q = 0.

Related concepts

Equality of survival functions test

Related tasks

Comparing two or more survival functions

Comparing two or more survival functions

Plot multiple survival curves to make comparisons between them.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Survival/Reliability, and then click Compare Survival Functions.
The analysis task pane opens.

3. In the Y (time to event) drop-down list, select the time until the event of interest variable.

4. Optional: In the Censor drop-down list, select the state variable that indicates whether the time was censored or if the event of interest occured, and then in the Event drop-down list, select the state that indicates the time was censored.

5. In the X (factor) drop-down list, select the grouping variable.

6. In the Survival function task panel, under the Equality of survival functions hypothesis test check box, in the Test drop-down list, select the statistical test to perform.

7. Click Calculate.

Related concepts

Kaplan-Meier survival function

Equality of survival functions test

Related reference

Tests for the equality of survival functions

Proportional hazards fit

A semi-parametric model that describes the relationship between a time to event response variable and one or more explanatory variables using a hazard function.

The Cox proportional hazard model is expressed as a hazard function h_0(t) + exp(∑βiXi). The formula expresses the hazard at time t for an individual with a given set of explanatory variables as the product of two quantities. The first quantity is h_0(t), the baseline hazard function. The second quantity is the exponent of the linear sum βiXi where the sum is over the p explanatory variables. An important feature of the proportional hazards model is that the baseline hazard function is a function of t but does not include the Xs. Therefore, it is unnecessary to specify the form of the baseline function in estimating the parameters. Also, the parameters in the second quantity are dependent only on the Xs and are independent of time. The unknown model parameters are estimated using maximum-likelihood estimation.

- Hazard ratio estimates
Hazard ratios are the increase or decrease in the hazard associated with a change of the predictor, all other predictors been held constant.

- Baseline survival function
A baseline survivual function is an estimate of h_0(t) in a proportional hazards model.

- Effect of model hypothesis test
A likelihood ratio or Wald X² test formally tests the hypothesis of whether the model fits the data better than no model.

- Effect of term hypothesis test
A likelihood ratio or Wald X² test formally tests the hypothesis of whether a term contributes to the model.

Related tasks

Fitting a proportional hazard model

Fitting a proportional hazard model

Fit a proportional hazards model to describe the relationship between a time to event response variable and one or more predictor variables.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Fit Model, and then click Proportional Hazard.
The analysis task pane opens.

3. In the Model drop-down menu, select Advanced.

4. In the Y drop-down list, select the response variable.

5. Optional. In the Censor drop-down list, select the censor variable, and then in the Event drop-down list, select the censor event. A censored event is an individual was lost to follow up or the event of interest had not occured at the end of the study.

6. In the Available variables list, select the predictor variable(s):

- To select a single variable, click the variable.

- To select multiple variables, click the first variable then hold down the CTRL key and click each additional variable.

- To select a range of variables, click the first variable then hold down the SHIFT key and click the last variable in the range.

7. Optional: Click the drop-down menu arrow next to the Available variables list, and then select the measurement scale of the variable.

If the measurement scale is not set, Analyse-it tries to determine if the variable is categorical or continuous scale. After you add a variable to a model, the icon alongside the variable indicates the assumed scale.

8. Click Add to add a variable, or click Factorial to add the factorial terms, or click Polynomial to add a polynomial terms.

9. To add the interaction between two or more variables, in the Terms list box, click the first term then hold down the CTRL key and click each additional term to include in the interaction, and then click Cross.

10. To remove a term, in the Terms list box, click the term and then click Remove.

11. Repeat steps 6 through 10 to build the model.

12. Click Calculate.

Related concepts

Hazard ratio estimates

Effect of model hypothesis test

Effect of term hypothesis test

Baseline survival function

Hazard ratio estimates

Hazard ratios are the increase or decrease in the hazard associated with a change of the predictor, all other predictors been held constant.

Although mathematically the log hazard ratios are easier to work with when fitting models, when it comes to interpretation, it is more natural to use the hazard ratios. A hazard ratio is defined as the hazard for one individual (or group) divided by the hazard for another individual or group. A value of 1 indicates no change in the hazard. Values between 0 and less than 1 indicate a decrease in the hazard . Values greater than 1 indicate an increase in the hazard.

When the predictor variable is a categorical variable, the hazard ratio is the increase or decrease in hazard over the baseline category.

When a predictor variable is a continuous variable, the hazard ratio is the increase or decrease in hazard for a change in the predictor variable. The default is for a 1 unit change in the predictor, although it may be more appropriate to use a larger unit, such as for a change of 10 units of the predictor variable.

When a term includes an interaction, the hazard ratio will depend on the interacting variables' values, so you will need to use multiple hazard ratios to describe the behavior at different levels of interest.

Related tasks

Estimating hazard ratios

Estimating hazard ratios

Estimate the hazard ratios using a proportional hazards model.

Ensure you have followed the steps in Fitting a proportional hazard model.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Fit Model group, click Hazard Ratios.
The analysis task pane Hazard Ratios panel opens.

3. In the Hazard Ratios grid, in the Comparison column.: For a continuous variable, type the change in units of the predictor for the hazard ratio, the default is for 1 unit change in the predictor, but for some predictors the odds ratio for a 10 or 100 unit change in the predictor maybe more interpretable. For a categorical variable, select the type of comparison:

Option

Description

All pairs

Compare all pairs of categories against each other. For example, with 3 categories the comparisons, are 1v2,1v3,2v1,2v3,3v1,3v2.

All distinct pairs

Compare all unique pairs of categories against each other. For example, with 3 categories the comparisons as 2v1,3v1,3v2).

Against reference

Compare all categories against the reference category (the first category in the categorical variable). For example, with 3 categories the comparisons are 3v1, 2v1.

4. If the model has interaction terms, in the At column for each continuous variable involved in an interaction, type the levels of interest as a comma (or your list delimiter) separated list. For example, if Age is a variable involved in an interaction with Gender, you may use 16, 25, 30, 40, and 60 as the levels of interest at which the Gender Female/Male hazard ratios will be computed. For a continuous variable involved in an interaction with a categorical variable, the unit change for each level of the categorical variable will be computed. For example, if the Age unit is 10 years and Age is involved in an interaction with Gender, the hazard ratio for a change of 10 years will be computed for Females and Males.

5. In the Confidence interval edit box, type the confidence level.

6. Click Recalculate.

Related concepts

Hazard ratio estimates

Baseline survival function

A baseline survivual function is an estimate of h_0(t) in a proportional hazards model.

A major advantage of the proportional hazard model is that it is semi-parametric, and baseline survival function h0(t) does not need to be specified to estimate the parameters. However, sometimes the baseline survival function is useful to understand the change in survival over time and combined with the estimated coefficients and specific covariate values to estimate the survival experience of subgroups of subjects of particular interest. The baseline survival function is estimated using maximum-likelihood at β=0 for the coefficients. In many cases, the baseline survival function for β=0 isn't easily visualized, and some recommend that the mean be subtracted before continuous variables are analyzed. In these cases, the reference survival function at β =0 for categorical variables (equivalent to the reference group) and β =mean for continuous variables is usually used.

Effect of model hypothesis test

A likelihood ratio or Wald X² test formally tests the hypothesis of whether the model fits the data better than no model.

It is common to test whether the model fits the data better than the null model with no parameters.

A X^2 test formally tests whether the reduction is statistically significant. The null hypothesis states that all the parameters for the covariates are zero against the alternative that at least one parameter is not equal to zero. When the p-value is small, you can reject the null hypothesis and conclude that at least one parameter is not zero.

Related concepts

Hypothesis testing

Proportional hazards fit

Effect of term hypothesis test

A likelihood ratio or Wald X² test formally tests the hypothesis of whether a term contributes to the model.

In most modeling analyses the aim is a model that describes the relationship using as few terms as possible. It is therefore of interest to look at each term in the model to decide if the term is providing any useful information.

A X^2 test for each term is a formal hypothesis test to determine if the term provides useful information to the model.  The null hypothesis states that the term does not contribute to the model, against the alternative hypothesis that it does. When the p-value is small, you can reject the null hypothesis and conclude that the term does contribute to the model.

When a term is not deemed to contribute statistically to the model, you may consider removing it. However, you should be cautious of removing terms that are known to contribute by some underlying mechanism, regardless of the statistical significance of a hypothesis test, and recognize that removing a term can alter the effect of other terms.

Related concepts

Hypothesis testing

Proportional hazards fit

Chapter 23. Control charts

Control charts determine if a process is in a state of statistical control.

A control chart plots a quality characteristic statistic in a time-ordered sequence. A center line indicates the process average, and two other horizontal lines called the lower and upper control limits represent process variation.

All processes have some natural degree of variation. A control chart for a process that is in-control has points randomly distributed within the control limits. That is it has variation only from sources common to the process (called common-cause variation). An out-of-control process has points falling outside the control limits or non-random patterns of points (called special-cause variation).

If the process is in-control, no corrections or changes to the process are needed.

If the process is out-of-control, the control chart can help determine the sources of variation in need of further investigation. It is appropriate to determine if the results with the special-cause are better than or worse than results from common causes alone. If worse, then that cause should be eliminated if possible. If better, it may be appropriate to investigate the system further as it may lead to improvements in the process.

Typically control limits are defined as a multiple of the process sigma. For a Shewhart control chart with 3-sigma control limits and assuming normality, the probability of exceeding the upper control limit is 0.00135 and the probability of falling below the lower control limit is also 0.00135. Their sum is 0.0027 (0.27%). Therefore the probability of a point between the control limits for an in-control process is 0.9973 (99.73%). An alternative is to define the control limits as probability limits based on a specified distribution rather than assuming a normal distribution.

Another way to look at the performance of a control chart is the average run length (ARL). An average in control run length is the number of observations when a process is in-control before a false alarm occurs. An average out of control run length is the number of observations that a process is out-of-control before a shift is detected, and depends on the size of the shift to be detected. The Shewhart control chart described above has an ARL = 1/0.0027 = 370.37. That is, when a process is in control, you should expect a false alarm out-of-control signal approximately once every 371 runs.

Although most examples of control charts show quality characteristics that are of interest to the end-user (such as length, diameter, or weight) they are most beneficial applied to process variables further upstream (such as the temperature of the furnace or content of tin in the raw material).

Note: It is important not to confuse control limits used in control charts with specification limits used in process capability. Natural variation in a process defines the control limits. Whereas, customer requirements define the specification limits. Likewise, the center line should not be confused with a target value.

- Shewhart control charts
A Shewhart control chart detects changes in a process.

- Time-weighted control charts
A time-weighted control chart detects small shifts in a process mean.

Shewhart control charts

A Shewhart control chart detects changes in a process.

Basic Shewhart control chart

Advanced Shewhart control chart with phases and stratification

- Shewhart variables control charts
Variables control charts plot quality characteristics that are numerical (for example, weight, the diameter of a bearing, or temperature of the furnace).

- Shewhart attributes control charts
Attributes control charts plot quality characteristics that are not numerical (for example, the number of defective units, or the number of scratches on a painted panel).

- Shewhart control chart rules
Tests for special-cause variation determine when a process needs further investigation.

Shewhart variables control charts

Variables control charts plot quality characteristics that are numerical (for example, weight, the diameter of a bearing, or temperature of the furnace).

There are two types of variables control charts: charts for data collected in subgroups, and charts for individual measurements. For sub-grouped data, the points represent a statistic of subgroups such as the mean, range, or standard deviation. For individual measurements, the points represent the individual observations or a statistic such as the moving range between successive observations.

An important concept for data collected in subgroups is that of a rational subgroup. A rational subgroup is a number of units produced under the same set of conditions. The measurements in each subgroup should be taken within a short period and should minimize the chances of special-cause variation occurring between the observations. That is, it should represent a snapshot of the process and the inherent common-cause variation.

Subgroups are often preferable when learning about a process as they can better estimate the process variability or detect smaller shifts in the process average. However, when monitoring a process, if the amount of time taken to collect enough observations to form a subgroup is too long, or the cost of sampling is expensive or destructive then the benefits may be negated.

- Xbar-R chart
An Xbar-R chart is a combination of control charts used to monitor the process variability (as the range) and average (as the mean) when measuring subgroups at regular intervals from a process.

- Xbar-S chart
An Xbar-S chart is a combination of control charts used to monitor the process variability (as the standard deviation) and average (as the mean) when measuring subgroups at regular intervals from a process.

- Xbar chart
An Xbar-chart is a type of control chart used to monitor the process mean when measuring subgroups at regular intervals from a process.

- R chart
An R-chart is a type of control chart used to monitor the process variability (as the range) when measuring small subgroups (n ≤ 10) at regular intervals from a process.

- S chart
An S-chart is a type of control chart used to monitor the process variability (as the standard deviation) when measuring subgroups (n ≥ 5) at regular intervals from a process.

- I-MR chart
An I-MR chart is a combination of control charts used to monitor the process variability (as the moving range between successive observations) and average (as the mean) when measuring individuals at regular intervals from a process.

- I chart
An I-chart is a type of control chart used to monitor the process mean when measuring individuals at regular intervals from a process.

- MR chart
An MR-chart is a type of control chart used to process variability (as the moving range of successive observations) when measuring individuals at regular intervals from a process.

Xbar-R chart

An Xbar-R chart is a combination of control charts used to monitor the process variability (as the range) and average (as the mean) when measuring subgroups at regular intervals from a process.

- Xbar chart
An Xbar-chart is a type of control chart used to monitor the process mean when measuring subgroups at regular intervals from a process.

- R chart
An R-chart is a type of control chart used to monitor the process variability (as the range) when measuring small subgroups (n ≤ 10) at regular intervals from a process.

Related tasks

Creating a Xbar-R / S control chart

Xbar-S chart

An Xbar-S chart is a combination of control charts used to monitor the process variability (as the standard deviation) and average (as the mean) when measuring subgroups at regular intervals from a process.

- Xbar chart
An Xbar-chart is a type of control chart used to monitor the process mean when measuring subgroups at regular intervals from a process.

- S chart
An S-chart is a type of control chart used to monitor the process variability (as the standard deviation) when measuring subgroups (n ≥ 5) at regular intervals from a process.

Related tasks

Creating a Xbar-R / S control chart

Xbar chart

An Xbar-chart is a type of control chart used to monitor the process mean when measuring subgroups at regular intervals from a process.

Each point on the chart represents the value of a subgroup mean.

The center line is the process mean. If unspecified, the process mean is the weighted mean of the subgroup means.

The control limits are either:

- A multiple (k) of sigma above and below the center line. Default k=3.

- Probability limits, defined as the probability (alpha) of a point exceeding the limits. Default alpha=0.27%.

If unspecified, the process sigma is the pooled standard deviation of the subgroups, unless the chart is combined with an R- or S- chart where it is estimated as described for the respective chart.

The observations are assumed to be independent, and the means normally distributed. Individual observations need not be normally distributed. Due to the central limit theorem, the subgroup means are often approximately normally distributed for subgroup sizes larger than 4 or 5 regardless of the distribution of the individual observations.

For data with different subgroup sizes, the control limits vary. A standardized version of the control chart plots the points in standard deviation units. Such a control chart has a constant center line at 0, and upper and lower control limits of -3 and +3 respectively making patterns in the data easier to see.

It is important to ensure process variability is in a state of statistical control before using the Xbar-chart to investigate if the process mean is in control. Therefore a Xbar-chart is often combined with an R- or S-chart to monitor process variability. If the variability is not under control, the control limits may be too wide leading to an inability to detect special causes of variation affecting the process mean.

Related concepts

R chart

S chart

Related tasks

Creating a Xbar-R / S control chart

Related information

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Ryan, T. P. (2011). Statistical methods for quality improvement. John Wiley & Sons.

R chart

An R-chart is a type of control chart used to monitor the process variability (as the range) when measuring small subgroups (n ≤ 10) at regular intervals from a process.

Each point on the chart represents the value of a subgroup range.

The center line for each subgroup is the expected value of the range statistic. Note that the center line varies when the subgroup sizes are unequal.

The control limits are either:

- A multiple (k) of sigma above and below the center line. Default k=3.

- Probability limits, defined as the probability (alpha) of a point exceeding the limits. Default alpha=0.27%.

If unspecified, the process sigma is the weighted average of the unbiased subgroup estimates of sigma based on the range statistics.

For small subgroup sizes with k-sigma control limits the lower limit is zero in many situations due to the asymmetric distribution of the range statistic. In these cases, probability limits may be more appropriate.

Related concepts

S chart

Related tasks

Creating a Xbar-R / S control chart

Related information

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Ryan, T. P. (2011). Statistical methods for quality improvement. John Wiley & Sons.

S chart

An S-chart is a type of control chart used to monitor the process variability (as the standard deviation) when measuring subgroups (n ≥ 5) at regular intervals from a process.

Each point on the chart represents the value of a subgroup standard deviation.

The center line for each subgroup is the expected value of the standard deviation statistic. Note that the center line varies when the subgroup sizes are unequal.

The control limits are either:

- A multiple (k) of sigma above and below the center line. Default k=3.

- Probability limits, defined as the probability (alpha) of a point exceeding the limits. Default alpha=0.27%.

If unspecified, the process sigma is the weighted average of the unbiased subgroup estimates of sigma based on the standard deviation statistics.

The individual observations are assumed to be independent and normally distributed. Although the chart is fairly robust and non-normality is not a serious problem unless there is a considerable deviation from normality (Burr, 1967).

For subgroup sizes less than 6 with k-sigma control limits, the lower limit is zero. In these cases, probability limits may be more appropriate.

Related concepts

R chart

Related tasks

Creating a Xbar-R / S control chart

Related information

Burr, I. W. (1967). The effect of non-normality on constants for Xbar and R charts. Industrial Quality Control, 23(11), 563-569.

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Ryan, T. P. (2011). Statistical methods for quality improvement. John Wiley & Sons.

I-MR chart

An I-MR chart is a combination of control charts used to monitor the process variability (as the moving range between successive observations) and average (as the mean) when measuring individuals at regular intervals from a process.

- I chart
An I-chart is a type of control chart used to monitor the process mean when measuring individuals at regular intervals from a process.

- MR chart
An MR-chart is a type of control chart used to process variability (as the moving range of successive observations) when measuring individuals at regular intervals from a process.

Related tasks

Creating an I-MR control chart

I chart

An I-chart is a type of control chart used to monitor the process mean when measuring individuals at regular intervals from a process.

It is typical to use individual observations instead of rational subgroups when there is no basis for forming subgroups, when there is a long interval between observations becoming available, when testing is destructive or expensive, or for many other reasons. Other charts such as the exponentially weighted moving average and cumulative sum may be more appropriate to detect smaller shifts more quickly.

Each point on the chart represents the value of an individual observation.

The center line is the process mean. If unspecified, the process mean is the mean of the individual observations.

The control limits are either:

- A multiple (k) of sigma above and below the center line. Default k=3.

- Probability limits, defined as the probability (alpha) of a point exceeding the limits. Default alpha=0.27%.

If unspecified, the process sigma is the standard deviation of the individual observations, unless the chart is combined with an MR-chart where it is estimated as described for the respective chart. For an in-control process, both estimators are unbiased, and the standard deviation is a more efficient estimator. For a process with special causes of variation, both estimators are biased, and it may be preferable to use the median moving range estimator.

The observations are assumed to be independent and normally distributed. If the process shows a moderate departure from normality, it may be preferable to transform the variable so that it is approximately normally distributed.

It is important to ensure process variability is in a state of statistical control before using the I-chart to investigate if the process mean is in control. Therefore an I-chart is often combined with an MR-chart to monitor process variability, although some authors question the benefit of an additional MR-chart.

Related concepts

MR chart

Related tasks

Creating an I-MR control chart

Related information

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Ryan, T. P. (2011). Statistical methods for quality improvement. John Wiley & Sons.

MR chart

An MR-chart is a type of control chart used to process variability (as the moving range of successive observations) when measuring individuals at regular intervals from a process.

When data are individual observations, it is not possible to use the standard deviation or range of subgroups to assess the variability and instead the moving range is an estimate the variability.

Given a series of observations and a fixed subset size, the first element of the moving range is the range of the initial subset of the number series. Then the subset is modified by "shifting forward"; that is, excluding the first number of the series and including the next number following the subset in the series. The next element of the moving range is the range of this subset. This process is repeated over the entire series creating a moving range statistic.

The moving range requires:

- A fixed subset size, the number of successive observations (span) in the moving range. Span must satisfy 1 < span ≤ n. Default span=2.
Larger values have a dampening effect on the statistic and may be preferable when the data are cyclical.

Each point on the chart represents the value of a moving range.

The center line is the expected value of the range statistic.

The control limits are either:

- A multiple (k) of sigma above and below the center line. Default k=3.

- Probability limits, defined as the probability (alpha) of a point exceeding the limits. Default alpha=0.27%.

If unspecified, the process sigma is the weighted average of the unbiased moving range estimates of sigma based on the range statistics.

You should be careful when interpreting a moving range chart because the values of the statistic are correlated. Correlation may appear as a pattern of runs or cycles on the chart. Some authors (Rigdon et al., 1994) recommend not plotting a moving range chart as moving range does not provide any useful information about shifts in process variability beyond the I-chart.

Related tasks

Creating an I-MR control chart

Related information

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Ryan, T. P. (2011). Statistical methods for quality improvement. John Wiley & Sons.

Rigdon, S. E., Cruthis, E. N., & Champ, C. W. (1994). Design strategies for individuals and moving range control charts. Journal of Quality Technology, 26(4), 274-287.

Creating a Xbar-R / S control chart

Plot a Shewhart control chart for data collected in rational subgroups to determine if a process is in a state of statistical control.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Process Control, and then click the type of control chart:

Option

Description

Xbar-R

Plot Shewhart control charts of the mean and range of subgroups.

Xbar-S

Plot Shewhart control charts of the mean and standard deviation of subgroups.

Xbar

Plot a Shewhart control chart of the mean of subgroups.

R

Plot a Shewhart control chart of the range of subgroups.

S

Plot a Shewhart control chart of the standard deviation of subgroups.

The analysis task pane opens.

3. In the Process drop-down list, select the quality characteristic variable.

4. In the Sample size group:

- If the subgroups are identified by a variable, select Subgroup by identifier variable, and in the Id drop-down list, select the subgroup identifier variable.

- If the subgroups are a constant size and formed by consecutive number of observations of the process variable, select Subgroup size constant, and in the edit box, enter the number of consecutive observations per subgroup.

5. In the Control limits drop-down list:

- To define control limits based on a multiple of the process sigma, select Sigma, and then in the edit box, type the multiple of the process sigma to define the control limits.

- To define control limits based on the probability distribution, select Probability, and then in the edit box, type the probability that a single statistic exceeds the control limits, assuming the process is in control.

6. Optional: To specify known process parameters, select the Known process parameters check box, and then in either or both the Mean and Standard deviation edit boxes, type the known value of the parameter.

7. Optional: To show the process mean and sigma statistics, select the Show process statistics check box.

8. Optional: To show the center and control limit statistics, select the Show process control statistics check box.

9. Click Calculate.

Related concepts

Xbar-S chart

Xbar chart

S chart

Xbar-R chart

R chart

Related tasks

Applying phases/stages for each change in a process

Applying rules to a Shewhart control chart

Formatting a control chart

Labeling control chart points

Related reference

Dataset layout

Creating an I-MR control chart

Plot a Shewhart control chart for individual observations to determine if a process is in a state of statistical control.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Process Control, and then click the type of control chart:

Option

Description

I-MR

Plot Shewhart control charts of the individual observations and moving range of consecutive observations.

I

Plot a Shewhart control chart of the individual observations.

MR

Plot a Shewhart control chart of the moving range of consecutive observations.

The analysis task pane opens.

3. In the Process drop-down list, select the quality characteristic variable.

4. Optional: In the Id drop-down list, select the identifier variable.

5. In the Control limits drop-down list:

- To define control limits based on a multiple of the process sigma, select Sigma, and then in the edit box, type the multiple of the process sigma to define the control limits.

- To define control limits based on the probability distribution, select Probability, and then in the edit box, type the probability that a single statistic exceeds the control limits, assuming the process is in control.

6. Optional: To specify known process parameters, select the Known process parameters check box, and then in either or both the Mean and Standard deviation edit boxes, type the known value of the parameter.

7. Optional: To show the process mean and sigma statistics, select the Show process statistics check box.

8. Optional: To show the center and control limit statistics, select the Show process control statistics check box.

9. Click Calculate.

Related concepts

I-MR chart

I chart

MR chart

Related tasks

Applying phases/stages for each change in a process

Applying rules to a Shewhart control chart

Formatting a control chart

Labeling control chart points

Related reference

Dataset layout

Shewhart attributes control charts

Attributes control charts plot quality characteristics that are not numerical (for example, the number of defective units, or the number of scratches on a painted panel).

It is sometimes necessary to simply classify each unit as either conforming or not conforming when a numerical measurement of a quality characteristic is not possible. In other cases, it is convenient to count the number of nonconformities rather than the number of nonconforming units. A unit may have a number of nonconformities without classing the unit as nonconforming. For example, a scratch on a painted panel may be a nonconformity but only if several such scratches exist would the entire panel be classed as a nonconforming unit.

- NP chart
A np-chart is a type of control chart used to monitor the number of nonconforming units when measuring subgroups at regular intervals from a process.

- P chart
A p-chart is a type of control chart used to monitor the proportion of nonconforming units when measuring subgroups at regular intervals from a process.

- C chart
A c-chart is a type of control chart used to monitor the total number of nonconformities when measuring subgroups at regular intervals from a process.

- U chart
A u-chart is a type of control chart used to monitor the average number of nonconformities per unit when measuring subgroups at regular intervals from a process.

NP chart

A np-chart is a type of control chart used to monitor the number of nonconforming units when measuring subgroups at regular intervals from a process.

Each point on the chart represents the number of nonconforming units in a subgroup.

The center line is the average number of nonconforming units. If unspecified, the process average proportion of nonconforming units is the total nonconforming units divided by the sum of subgroup sizes. Note that the center line varies when the subgroup sizes are unequal.

The control limits are either:

- A multiple (k) of sigma above and below the center line. Default k=3.

- Probability limits, defined as the probability (alpha) of a point exceeding the limits. Default alpha=0.27%.

A Binomial distribution is assumed. That is, units are either conforming or nonconforming, and that nonconformities are independent; the occurrence of a nonconforming unit at a particular point in time does not affect the probability of a nonconforming unit in the periods that immediately follow. Violation of this assumption can cause overdispersion; the presence of greater variance that would be expected based on the distribution. When k-sigma limits are used the normal approximation to the binomial distribution is assumed adequate which may require large subgroup sizes when the proportion of nonconforming units is small.

A np-chart is useful when the number of units in each subgroup is constant as interpretation is easier than a p-chart. For data with different subgroup sizes the center line and control limits both vary making interpretation difficult. In this case, you should use a p-chart which has a constant center line but varying control limits.

For small subgroup sizes, the lower control limit is zero in many situations. The lack of a lower limit is troublesome if the charts use is for quality improvement as the lower limit is desirable as points appearing below that value may reflect a significant reduction in the number of nonconforming units. It is necessary to increase the subgroup size to overcome this issue.

Related concepts

P chart

Related tasks

Creating an NP / P control chart

Related information

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Ryan, T. P. (2011). Statistical methods for quality improvement. John Wiley & Sons.

P chart

A p-chart is a type of control chart used to monitor the proportion of nonconforming units when measuring subgroups at regular intervals from a process.

A p-chart is a scaled version of the np-chart representing a proportion of nonconforming units rather than the number of nonconforming units. The same assumptions and recommendations apply.

For data with different subgroup sizes, the control limits vary although the center line is constant. A standardized version of the control chart plots the points in standard deviation units. Such a control chart has a constant center line at 0, and upper and lower control limits of -3 and +3 respectively making it easier to see patterns.

Related concepts

NP chart

Related tasks

Creating an NP / P control chart

Related information

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Ryan, T. P. (2011). Statistical methods for quality improvement. John Wiley & Sons.

C chart

A c-chart is a type of control chart used to monitor the total number of nonconformities when measuring subgroups at regular intervals from a process.

Each point on the chart represents the total number of nonconformities in a subgroup.

The center line is the average number of nonconformities. If unspecified, the process average number of nonconformities per unit is the total number of nonconformities divided by the sum of subgroup sizes. Note that the center line varies when the subgroup sizes are unequal.

The control limits are either:

- A multiple (k) of sigma above and below the center line. Default k=3.

- Probability limits, defined as the probability (alpha) of a point exceeding the limits. Default alpha=0.27%.

A Poisson distribution is assumed. That is, the probability of observing a nonconformity in the inspection unit should be small, but a large number of nonconformities should be theoretically possible, and the size of an inspection unit should also be constant over time. When k-sigma limits are used the normal approximation to the Poisson distribution is assumed adequate which usually requires the average number of nonconformities to be at least 5.

A c-chart is useful when the number of units in each subgroup is constant as interpretation is easier than a u-chart. For data with different subgroup sizes, the center line and control limits both vary making interpretation difficult. In this case, you should use a u-chart which has a constant center line but varying control limits.

Related concepts

U chart

Related tasks

Creating a C / U control chart

Related information

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Ryan, T. P. (2011). Statistical methods for quality improvement. John Wiley & Sons.

U chart

A u-chart is a type of control chart used to monitor the average number of nonconformities per unit when measuring subgroups at regular intervals from a process.

A u-chart is a scaled version of the c-chart representing the average number of nonconformities per unit rather than the number of nonconformities. The same assumptions and recommendations apply.

For data with different subgroup sizes, the control limits vary although the center line is constant. A standardized version of the control chart plots the points in standard deviation units. Such a control chart has a constant center line at 0, and upper and lower control limits of -3 and +3 respectively making it easier to see patterns.

Related concepts

C chart

Related tasks

Creating a C / U control chart

Related information

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Ryan, T. P. (2011). Statistical methods for quality improvement. John Wiley & Sons.

Creating an NP / P control chart

Plot a Shewhart control chart for the number/proportion of nonconforming units to determine if a process is in a state of statistical control.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Process Control, and then click the type of control chart:

Option

Description

np

Plot a Shewhart control chart of the number of nonconforming units.

p

Plot a Shewhart control chart of the proportion of nonconforming units.

The analysis task pane opens.

3. In the Process drop-down list, select the number of nonconforming units variable.

4. In the Sample size group:

- If the subgroup sizes are variable, select Sample size variable, and in the Size drop-down list, select the corresponding sample sizes variable.

- If the subgroups are a constant size, select Sample size constant, and in the edit box, enter sample size.

5. Optional: In the Id drop-down list, select the identifier variable.

6. In the Control limits drop-down list:

- To define control limits based on a multiple of the process sigma, select Sigma, and then in the edit box, type the multiple of the process sigma to define the control limits.

- To define control limits based on the probability distribution, select Probability, and then in the edit box, type the probability that a single statistic exceeds the control limits, assuming the process is in control.

7. Optional: To specify known process parameters, select the Known process parameters check box, and then in the Proportion edit box, type the known proportion of nonconforming units.

8. Optional: To show the process mean and sigma statistics, select the Show process statistics check box.

9. Optional: To show the center and control limit statistics, select the Show process control statistics check box.

10. Click Calculate.

Related concepts

NP chart

P chart

Related tasks

Applying phases/stages for each change in a process

Applying rules to a Shewhart control chart

Formatting a control chart

Labeling control chart points

Related reference

Dataset layout

Creating a C / U control chart

Plot a Shewhart control chart for the total number of nonconformities or the average number of nonconformities per unit to determine if a process is in a state of statistical control.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Process Control, and then click the type of control chart:

Option

Description

c

Plot a Shewhart control chart of the total number of nonconformities.

u

Plot a Shewhart control chart of the average number of nonconformities per unit.

The analysis task pane opens.

3. In the Process drop-down list, select the number of nonconformities variable.

4. In the Sample size group:

- If the subgroup sizes are variable, select Sample size variable, and in the Size drop-down list, select the corresponding sample sizes variable.

- If the subgroups are a constant size, select Sample size constant, and in the edit box, enter sample size.

5. Optional: In the Id drop-down list, select the identifier variable.

6. In the Control limits drop-down list:

- To define control limits based on a multiple of the process sigma, select Sigma, and then in the edit box, type the multiple of the process sigma to define the control limits.

- To define control limits based on the probability distribution, select Probability, and then in the edit box, type the probability that a single statistic exceeds the control limits, assuming the process is in control.

7. Optional: To specify known process parameters, select the Known process parameters check box, and then in the Mean edit box, type the known average number of nonconformities per unit.

8. Optional: To show the process mean and sigma statistics, select the Show process statistics check box.

9. Optional: To show the center and control limit statistics, select the Show process control statistics check box.

10. Click Calculate.

Related concepts

C chart

U chart

Related tasks

Applying phases/stages for each change in a process

Applying rules to a Shewhart control chart

Formatting a control chart

Labeling control chart points

Related reference

Dataset layout

Shewhart control chart rules

Tests for special-cause variation determine when a process needs further investigation.

There are various tests that can be used in conjunction with a control chart to identify special-cause variation:

Test

Rule

Problem indicated

1

1 point is outside the control limits.

A large shift.

2

8/9 points on the same size of the center line.

A small sustained shift.

3

6 consecutive points are steadily increasing or decreasing.

A trend or drift up or down.

4

14 consecutive points are alternating up and down.

Non-random systematic variation.

5

2 out of 3 consecutive points are more than 2 sigmas from the center line in the same direction.

A medium shift.

6

4 out of 5 consecutive points are more than 1 sigma from the center line in the same direction.

A small shift.

7

15 consecutive points are within 1 sigma of the center line.

Stratification.

8

8 consecutive points on either side of the center line with not within 1 sigma.

A mixture pattern.

You should choose tests in advance of looking at the control chart based on your knowledge of the process. Applying test 1 to a Shewhart control chart for an in-control process with observations from a normal distribution leads to a false alarm once every 370 observations on average. Additional tests make the chart more sensitive to detecting special-cause variation, but also increases the chance of false alarms. For example, applying tests 1, 2, 5, 6 raises the false alarm rate to once every 91.75 observations.

Most tests beyond test 1 are only appropriate when trying to bring a process under control. Tests 2, 3, 5, and 6 detect small shifts once a process is under control although it is often preferable to using a combination of a Shewhart chart with test 1 for detection of large shifts and an EWMA or CUSUM chart for detecting smaller shifts and trends.

It can be useful to add zones at ±1, 2, and 3 sigmas to the control chart to help interpret patterns. For control charts with unequal subgroup sizes, the center line, control limits and zones may vary. In this case, the tests apply to a standardized control chart where the points are the number of standard deviation units from the center line. Such a control chart has a constant center line at 0, and upper and lower control limits of +3 and -3 respectively making patterns easier to spot.

Tests 1, 5, 6, 2 are defined by the Western Electric CO (1958) as the original 4 rules. Tests 1-8, with the modification of test 2 to from 8 to 9 points, are defined by Lloyd S. Nelson (1984). Variations of these eight rules with different run lengths and rule ordering are recommended by various other authors, one of the most popular in the book by (Douglas C. Montgomery, 2012).

Related tasks

Applying rules to a Shewhart control chart

Related information

Nelson, L. S. (1984). Technical Aids: The Shewhart Control Chart -Tests for Special Causes. Journal of quality technology, 16(4).

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Western Electric Company. (1958). Statistical Quality Control Handbook. AT&T.

Applying rules to a Shewhart control chart

Apply advanced rules to increase the sensitivity of a Shewhart control chart to detect small shifts, drift, and other patterns.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Process Control group, click Apply Rules, and then click:

Option

Description

Standard

Apply the standard action rule only.

WECO

Apply the original 4 rules defined by the Western Electric CO.

Nelson

Apply the 8 rules suggested by Lloyd S. Nelson.

Montgomery

Apply the 8 Nelson rules as modified by Douglas C. Montgomery.

Custom

Apply a user-defined set of rules.

The analysis task pane Control Chart (Shewhart) panel opens.

3. Optional: To customize which rules to apply, select or clear the check boxes alongside the rules.

4. Optional: To change either or both the Run length, and Sigma in a custom ruleset, type the new value in the edit box(es) alongside the rules.

5. Optional: To standardize the quality characteristic statistic, so the control limits are constant regardless of the size of each subgroup, select the Standardized plot check box.

6. Optional: To show ±1,2,3 sigma zones for interpreting rules, select the Zones check box.

7. Click Recalculate.

A red asterisk marks points on the control chart that break a rule. A label indicates the broken rule number. Line segments between the points that form part of the broken rule are colored orange.
Note: Rule numbering depends on the ruleset applied. Only rule 1 is the same regardless of the ruleset.

Related concepts

Shewhart control chart rules

Formatting a control chart

Change the style of the control chart.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Process Control group, under the Format heading, click Style, and then click:

Option

Description

Point

Show only points.

Line

Show only connecting lines, except for out-of-control points.

Point+Line

Show points and connecting lines.

The analysis task pane Control Chart panel opens.

3. To show the individual subgroup observations where applicable to the control chart, select the Individual observations check box, and then in the drop-down list select:

Option

Description

Dot

Show individual observations

High-Low

Show a box spanning the highest to lowest observations.

Boxplot

Show a boxplot of the observations.

4. Click Recalculate.

Labeling control chart points

Label control chart points to identify them or document causes of special-cause variation.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Process Control group, under the Format heading, click Label.
The analysis task pane Control Chart panel opens.

3. Alongside the Label points check box, select:

Option

Description

All

Label all points.

Out-of-control

Label only points that are out-of-control.

4. In the Label drop-down list, select a variable.

5. Click Recalculate.

Displaying stratification in a control chart

Stratify the points on a control chart using colors and symbols to uncover causes of variation the process.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Process Control group, click Stratification.
The analysis task pane Control Chart panel opens, and the Vary point color/symbol check box is selected.

3. In the Color / Symbol drop-down list, select a variable.

4. Click Recalculate.

Related tasks

Assigning colors/symbols to categories

Applying phases/stages for each change in a process

Use separate process statistics for different phases/stages or when other changes occur in a process that affects the process quality characteristic statistic.

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Process Control group, click Phases.

3. In the By drop-down list, select the phase/stage, or another change variable.

4. Click Recalculate.

Related information

Estimating process capability

Time-weighted control charts

A time-weighted control chart detects small shifts in a process mean.

Time-weighted control charts combine information from the current and past observations. They are more sensitive to detecting small shifts in the process mean than a Shewhart chart, but slower at detecting large shifts.

The charts only monitor the process mean; monitoring the process variability requires the use of some other technique.

- Uniformly Weighted Moving Average (UWMA) chart
A uniformly weighted moving average (UWMA) chart is a type of control chart used to monitor small shifts in the process mean. It uses the average of a number of consecutive observations.

- Exponentially Weighted Moving Average (EWMA) chart
An exponentially weighted moving average (EWMA) chart is a type of control chart used to monitor small shifts in the process mean. It weights observations in geometrically decreasing order so that the most recent observations contribute highly while the oldest observations contribute very little.

- CUmulative SUM (CUSUM) chart
A cumulative sum (CUSUM) chart is a type of control chart used to monitor small shifts in the process mean. It uses the cumulative sum of deviations from a target.

Uniformly Weighted Moving Average (UWMA) chart

A uniformly weighted moving average (UWMA) chart is a type of control chart used to monitor small shifts in the process mean. It uses the average of a number of consecutive observations.

The UWMA chart plots the moving average for individual measurements or subgroup means.

Given a series of observations and a fixed subset size, the first element of the moving average is the average of the initial subset of the number series. Then the subset is modified by "shifting forward"; that is, excluding the first number of the series and including the next number following the subset in the series. The next element of the moving average is the average of this subset. This process is repeated over the entire series creating the moving average statistic.

The UWMA requires:

- A fixed subset size, the number of successive observations (span) in the moving average. Span must satisfy 1 < span ≤ n. Default span=3.
A small span reduces the influence of older observations; a large span slows the response to large shifts. In general, the magnitude of the shift to detect and the span are inversely related.

Each point on the chart represents the value of a moving average.

The center line is the process mean. If unspecified, the process mean is the weighted mean of the subgroup means or the mean of the individual observations.

The control limits are a multiple (L) of sigma above and below the center line. Default L=3. If unspecified, the process sigma is the pooled standard deviation of the subgroups, or the standard deviation of the individual observations, unless the chart is combined with an R-, S-, or MR- chart where it is estimated as described for the respective chart.

UWMA is sensitive to small shifts in the process mean, but is not as effective as either the CUSUM or EWMA (Montgomery 2012).

Related concepts

Exponentially Weighted Moving Average (EWMA) chart

Related tasks

Creating a UWMA control chart

Related information

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Exponentially Weighted Moving Average (EWMA) chart

An exponentially weighted moving average (EWMA) chart is a type of control chart used to monitor small shifts in the process mean. It weights observations in geometrically decreasing order so that the most recent observations contribute highly while the oldest observations contribute very little.

The EWMA chart plots the exponentially weighted moving average of individual measurements or subgroup means.

Given a series of observations and a fixed weight, the first element of the exponentially weighted moving average is computed by taking the (1-weight) * previous EWMA + (weight * current observation). Then the current observation is modified by "shifting forward"; and repeating the calculation. This process is repeated over the entire series creating the exponentially weighted moving average statistic.

The EWMA requires:

- A weight for the most recent observation. Weight must satisfy 0 < weight ≤ 1. Default weight=0.2.
The "best" value is a matter of personal preference and experience. A small weight reduces the influence of the most recent sample; a large value increases the influence of the most recent sample. A value of 1 reduces the chart to a Shewhart Xbar chart. Recommendations suggest a weight between 0.05 and 0.25 (Montgomery 2012).

When designing an EWMA chart it is necessary to consider the average run length and shift to be detected. Extensive guidance is available on suitable parameters (Montgomery 2012).

It is possible to modify the EWMA, so it responds more quickly to detect a process that is out-of-control at start-up. This modification is done using a further exponentially decreasing adjustment to narrow the limits of the first few observations (Montgomery 2012).

Each point on the chart represents the value of the exponentially weighted moving average.

The center line is the process mean. If unspecified, the process mean is the weighted mean of the subgroup means or the mean of the individual observations.

The control limits are a multiple (L) of sigma above and below the center line. Default L=3. If unspecified, the process sigma is the pooled standard deviation of the subgroups, or the standard deviation of the individual observations, unless the chart is combined with an R-, S-, or MR- chart where it is estimated as described for the respective chart.

Because the EWMA is a weighted average of all past and the current observations, it is very insensitive to the assumption of normality. It is, therefore, an ideal replacement for a Shewhart I-chart when normality cannot be assumed.

Like the CUSUM, EWMA is sensitive to small shifts in the process mean but does not match the ability of a Shewhart chart to detect larger shifts. For this reason, it is sometimes used together with a Shewhart chart (Montgomery 2012).

Related concepts

CUmulative SUM (CUSUM) chart

Shewhart variables control charts

Related tasks

Creating an EWMA control chart

Related information

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Creating a UWMA control chart

Plot a UWMA control chart to determine if a process is in a state of statistical control.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Process Control, and then click UWMA
The analysis task pane opens.

3. In the Model drop-down menu:

- If the data are individual observations, select Process variable .

- If the data are rational subgroups, select Process variable with Subgroups.

4. In the Process drop-down list, select the quality characteristic variable.

5. If the data are collected in subgroups, in the Sample size group:

- If the subgroups are identified by a variable, select Subgroup by identifier variable, and in the Id drop-down list, select the subgroup identifier variable.

- If the subgroups are a constant size and formed by consecutive number of observations of the process variable, Subgroup size constant, and in the edit box, enter the number of consecutive observations per subgroup.

6. Optional: If the data are individual observations, in the Id drop-down list, select an identifier variable

7. In the Span edit box, type the number of consecutive values to compute the UWMA statistic.

8. In the L edit box, type the multiple of the process sigma to define the control limits.

9. Optional: To reset the value of the UWMA statistic after each out-of-control signal, select the Reset after out-of-control signal check box.

10. Optional: To specify known process parameters, select the Known process parameters check box, and then in either or both the Mean and Standard deviation edit boxes, type the known value of the parameter.

11. Optional: To show the process mean and sigma statistics, select the Show process statistics check box.

12. Optional: To show the center and control limit statistics, select the Show process control statistics check box.

13. Click Calculate.

Related concepts

Uniformly Weighted Moving Average (UWMA) chart

Related tasks

Applying phases/stages for each change in a process

Formatting a control chart

Labeling control chart points

Related reference

Dataset layout

Creating an EWMA control chart

Plot an EWMA control chart to determine if a process is in a state of statistical control.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Process Control, and then click EWMA
The analysis task pane opens.

3. In the Model drop-down menu:

- If the data are individual observations, select Process variable .

- If the data are rational subgroups, select Process variable with Subgroups.

4. In the Process drop-down list, select the quality characteristic variable.

5. If the data are collected in subgroups, in the Sample size group:

- If the subgroups are identified by a variable, select Subgroup by identifier variable, and in the Id drop-down list, select the subgroup identifier variable.

- If the subgroups are a constant size and formed by consecutive number of observations of the process variable, Subgroup size constant, and in the edit box, enter the number of consecutive observations per subgroup.

6. Optional: If the data are individual observations, in the Id drop-down list, select an identifier variable

7. In the Weight edit box, type the weight to be assigned to the current sample in the EWMA statistic.

8. In the L edit box, type the multiple of the process sigma to define the control limits.

9. Optional: To detect existing out-of-control situations quickly, select the Headstart check box, and in the a and f edit boxes, type the parameters.
A value of a=0.3 and f=0.5 mimics a 50% headstart.

10. Optional: To reset the value of the EWMA statistic after each out-of-control signal, select the Reset after out-of-control signal check box.

11. Optional: To specify known process parameters, select the Known process parameters check box, and then in either or both the Mean and Standard deviation edit boxes, type the known value of the parameter.

12. Optional: To show the process mean and sigma statistics, select the Show process statistics check box.

13. Optional: To show the center and control limit statistics, select the Show process control statistics check box.

14. Click Calculate.

Related concepts

Exponentially Weighted Moving Average (EWMA) chart

Related tasks

Applying phases/stages for each change in a process

Formatting a control chart

Labeling control chart points

Related reference

Dataset layout

CUmulative SUM (CUSUM) chart

A cumulative sum (CUSUM) chart is a type of control chart used to monitor small shifts in the process mean. It uses the cumulative sum of deviations from a target.

The CUSUM chart plots the cumulative sum of deviations from the target for individual measurements or subgroup means.

The CUSUM requires two parameters:

- A reference value (k) specified in sigma units. k is often set to half the shift to be detected, in sigma units. Default k=0.5 which is equal to detecting a shift of 1 sigma.

- The decision limit (h) specified in sigma units. Default h=5.

When designing a CUSUM chart it is necessary to consider the average run length and shift to be detected. Extensive guidance is available on suitable parameters (NIST 2012, Montgomery 2012).

It is possible to modify the CUSUM, so it responds more quickly to detect a process that is out-of-control at start-up. This modification is done using an initial CUSUM equal to half of the h parameter.

Like the EWMA, CUSUM is sensitive to small shifts in the process mean but does not match the ability of a Shewhart chart to detect larger shifts. For this reason, it is sometimes used together with a Shewhart chart (Montgomery 2012).

Related concepts

Exponentially Weighted Moving Average (EWMA) chart

Shewhart variables control charts

Related tasks

Creating a CUSUM control chart

Related information

N. I. S. T. (2012). e-Handbook of Statistical Methods.

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Creating a CUSUM control chart

Plot a CUSUM control chart to determine if a process is in a state of statistical control.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Process Control, and then click CUSUM
The analysis task pane opens.

3. In the Model drop-down menu:

- If the data are individual observations, select Process variable .

- If the data are rational subgroups, select Process variable with Subgroups.

4. In the Process drop-down list, select the quality characteristic variable.

5. If the data are collected in subgroups, in the Sample size group:

- If the subgroups are identified by a variable, select Subgroup by identifier variable, and in the Id drop-down list, select the subgroup identifier variable.

- If the subgroups are a constant size and formed by consecutive number of observations of the process variable, Subgroup size constant, and in the edit box, enter the number of consecutive observations per subgroup.

6. Optional: If the data are individual observations, in the Id drop-down list, select an identifier variable

7. In the k and h edit boxes, type the design parameters for the CUSUM chart.

Note: k and h are specified in multiples of sigma and not in the data measurement units.

8. Optional: To detect existing out-of-control situations quickly, select the Headstart check box, and in the Headstart h edit box, type the headstart value.
Typically the headstart h is set to half the h value.

9. Optional: To standardize the CUSUM statistic, so the control limits are constant regardless of the size of each subgroup, select Standardized plot.

10. Optional: To reset the value of the CUSUM statistic after each out-of-control signal, select the Reset after out-of-control signal check box.

11. Optional: To specify known process parameters, select the Known process parameters check box, and then in either or both the Mean and Standard deviation edit boxes, type the known value of the parameter.

12. Optional: To show the process mean and sigma statistics, select the Show process statistics check box.

13. Optional: To show the center and control limit statistics, select the Show process control statistics check box.

14. Click Calculate.

Related concepts

CUmulative SUM (CUSUM) chart

Related tasks

Applying phases/stages for each change in a process

Formatting a control chart

Labeling control chart points

Related reference

Dataset layout

Study design

Process control and capability analysis study requirements and dataset layout.

Requirements

- A categorical or quantitative variable.

- A stratification variable.

- A subgroup variable.

- A phase/stage variable.

Dataset layout for individuals or fixed subgroup size

Use a column for each variable (Copper); each row has a single measurement.

Copper

7.96

8.52

9.24

7.96

10.04

8.68

7.46

8.84

8.9

9.28

…

Advanced dataset layout with stratification and phases

Use a column for each variable (Copper), and optionally columns for subgroup (Day), stratification (Operator), phase/stage (Phase); each row has a single measurement.

Day (optional)

Operator (optional)

Phase (optional)

Copper

Comments (optional)

1

SNH

IQ

7.96

1

SNH

IQ

8.52

2

JDH

IQ

9.24

2

GMH

IQ

7.96

3

SNH

IQ

10.04

Electrode failure

3

SNH

IQ

8.68

…

…

IQ

…

20

JDH

OQ

8.68

20

SNH

OQ

8.76

21

JDH

OQ

8.02

21

GMH

OQ

8.7

…

…

OQ

…

Frequency form dataset layout

Use a column for each subgroup (Sample), a column for the subgroup size (Sample size), and a column for the number of cases (Frequency); each row has a the frequency count for each sample.

Sample

Sample size

Nonconforming units

1

100

7

2

80

8

3

80

12

4

100

6

5

110

10

6

110

12

7

100

16

8

90

10

9

90

6

10

120

20

…

…

…

Related information

Estimating process capability

Chapter 24. Process capability

Capability analysis measures the ability of a process to meet specifications when the process is in statistical control.

A process must be in control before attempting to assess the capability. An out-of-control process is unpredictable and not capable of been characterized by a probability distribution.

Most process capability indices assume a normally distributed quality characteristic. If the distribution is non-normal, it may be possible to transform the data to be normally distributed. The process mean and process sigma define the normal distribution.

Capability indices are either "long-term" or "short-term" depending on the definition of the process sigma:

- Long-term indices measure the process performance and represent the quality the end-user experiences. They are computed using the process sigma that includes both within-subgroup and between-subgroup variation (the standard deviation of the individual measurements).

- Short-term indices measure the potential process performance ignoring differences between subgroups. They are computed using the process sigma that includes only within-subgroup variation (the Xbar-, R-, S-, or MR- control chart process sigma).

If the process is stable over time, the estimates of short-term sigma and long-term sigma are very similar. They are both estimates of the same parameter, although statistically speaking the long-term sigma is a slightly more efficient estimator.

However, if there are any changes in the process mean over time, the estimate of long-term sigma is greater than that of short-term sigma. The larger the difference between the values of long-term and short-term indices, the more opportunity there is to improve the process by eliminating drift, shifts and other sources of variation.

Note: There is much confusion over the meaning of the phrases long-term and short-term. It is important not to confuse them with the collection period of the sample data.

- Capability ratios (Cp/Pp indices)
Capability ratios (Cp/Pp) describe the variability of a process relative to the specification limits.

- Z benchmark
Z benchmark describes the sigma capability of a process.

- Nonconforming units
Nonconforming units describe the number of nonconforming units a process produces, expressed in parts per million.

Capability ratios (Cp/Pp indices)

Capability ratios (Cp/Pp) describe the variability of a process relative to the specification limits.

A capability ratio is a unit-less value describing the ratio of process distribution spread to specification limits spread. A value of less than 1 is unacceptable, with values greater than 1.33 (1.25 for one-sided specification limits) widely accepted as the minimum acceptable value, and values greater than 1.50 (1.45 for one-sided specification limits) for critical parameters (Montgomery, 2012). The higher the value, the more capable the process of meeting specifications. A value of 2 or higher is required to achieve Six Sigma capability which is defined as the process mean not closer than six standard deviations from the nearest specification limit.

All of the indices assume a normally distributed process quality characteristic with the parameters specified by the process mean and sigma. The process sigma is either the short-term or long-term sigma estimate.

Indices computed using the short-term sigma estimate are called Cp indices (Cp, Cpl, Cpu, Cpk, Cpm). While those using long-term sigma estimate are called Pp indices (Pp, Ppl, Ppu, Ppk, Ppm). If the Cp indices are much smaller than the Pp indices, it indicates that there are improvements you could make by eliminating shifts and drifts in the process mean.

Various indices measure how the process is performing against the specification limits:

Index

Purpose

Cp/Pp

Estimates the capability of a process if the process mean were to be centered between the specification limits.
Note: If the process mean is not centered between the specification limits the value is only the potential capability, and you should not report it as the capability of the process.

Cpl/Ppl

Estimates the capability of a process to meet the lower specification limit. Defined as how close the process mean is to the lower specification limit.

Cpu/Ppu

Estimates the capability of a process to meet the upper specification limit. Defined as how close the process mean is to the upper specification limit.

Cpk/Ppk

Estimates the capability of a process, considering that the process mean may not be centered between the specification limits. Defined as the lesser of Cpl and Cpu.
Note: If Cpk is equal to Cp, then the process is centered at the midpoint of the specification limits. The magnitude of Cpk relative to Cp is a measure of how off center the process is and the potential improvement possible by centering the process.

Cpm/Ppm

Estimates the capability of a process, and is dependent on the deviation of the process mean from the target.
Note: Cpm increases as the process mean moves towards the target. Cpm, Cpk, and Cp all coincide when the target is the center of the specification limits and the process mean is centered.

Note: There is some confusion between terms "Cp" and "Pp" as some authors suggest the use of Pp indices when a process is not-in-control and Cp indices when a process is in control. However, it is nonsense to interpret the indices when the process is not-in-control as no probability distribution can describe the process performance. We make the distinction between Pp and Cp indices on the estimate of sigma used not on the state of the process.

Related tasks

Estimating process capability

Related information

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Z benchmark

Z benchmark describes the sigma capability of a process.

Z benchmark indices are an alternative to Cp and Pp indices. They are the definition of the sigma capability of a Six Sigma process.

All of the indices assume a normally distributed process quality characteristic with the parameters specified by the process mean and sigma. The process sigma is either the short-term or long-term sigma estimate.

Various indices measure how the process is performing against the specification limits:

Index

Description

< LSL

The number of sigma units from the process mean to the lower specification limit.

> USL

The number of sigma units from the process mean to the upper specification limit.

< SL >

The number of sigma units from the process to mean to the point if all nonconforming units are put in one tail of the distribution.

Z shift is the difference between the short-term and long-term indices. The larger the Z shift, the more scope there is to improve the process by eliminating shifts and drifts in the process mean. Some industries define the sigma capability of a process as the long-term Z benchmark + a 1.5 Z shift. Meaning a process with a long-term Z benchmark of 4.5 is quoted as a Six Sigma process. It is best to avoid such rules and directly measure the short-term and long-term Z benchmark capability.

Related tasks

Estimating process capability

Nonconforming units

Nonconforming units describe the number of nonconforming units a process produces, expressed in parts per million.

The number of nonconforming units is an alternative to traditional Cp and Pp indices. They are easily understandable by end-users.

The number of nonconforming units is either:

- The actual number of nonconforming units in the sample.

- The expected number of nonconforming units, assuming a normally distributed process quality characteristic with the parameters specified by the process mean and sigma. The process sigma is either the short-term or long-term sigma estimate.

Various indices measure how the process is performing against the specification limits:

Index

Description

< LSL

The number of nonconforming units that are less than the lower specification limit.

> USL

Thr number of nonconforming units that are greater than the upper specification limit.

< SL >

The total number of nonconforming units that are outside the specification limits.

Related tasks

Estimating process capability

Estimating process capability

Plot a process capability histogram and compute indices to determine if a process can meet specifications.

This task creates a new process capability analysis report. You can also estimate the process capability directly from a process control analysis report using the Capability command in the Process Capability group on the Analyse-it ribbon tab.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Capability, and then click:

Option

Description

3-up

Compute capability ratios and plot a histogram and univariate plot.

4-up

Compute capability ratios and plot a histogram, univariate plot, and normal probability plot.

6-up

Compute capability ratios and plot a histogram, univariate plot, normal probability plot and control charts.

The analysis task pane opens.

3. In the Model drop-down menu:

- If the data are individual observations, select Process variable .

- If the data are rational subgroups, select Process variable with Subgroups.

4. In the Process drop-down list, select the quality characteristic variable.

5. If the data are collected in subgroups, in the Sample size group:

- If the subgroups are identified by a variable, select Subgroup by identifier variable, and in the Id drop-down list, select the subgroup identifier variable.

- If the subgroups are a constant size and formed by consecutive number of observations of the process variable, Subgroup size constant, and in the edit box, enter the number of consecutive observations per subgroup.

6. Optional: If the data are individual observations, in the Id drop-down list, select an identifier variable

7. In either or both the LSL and USL edit boxes, type the lower and upper specification limits.

8. Optional: In the Target edit box, type the target value.

9. Optional: If you are using control charts, in the Sigma estimator drop-down list, select:

Option

Description

Long-term

Use the overall standard deviation of the individual measurements.

Short-term

Use the within-subgroup sigma of the process (using the process control R-, S-, or MR- chart).

10. Select the Capability ratios, Z benchmark, or Nonconforming units check boxes.

11. Click Calculate.

Related concepts

Capability ratios (Cp/Pp indices)

Nonconforming units

Z benchmark

Related tasks

Applying phases/stages for each change in a process

Related reference

Dataset layout

Chapter 25. Pareto analysis

Pareto analysis identifies the most important quality-related problems to resolve in a process.

The Pareto principle (also known as the 80/20 rule) states that for many events, roughly 80% of the problems come from 20% of the causes. This statement is merely a rule of thumb and is not an immutable law of nature. More generally, a small subset of issues tend to cause most problems, and it is useful to identify those issues as they have the most impact on the process.

- Pareto chart
A Pareto chart shows the frequency of occurrences of quality-related problems to highlight those that need the most attention.

Pareto chart

A Pareto chart shows the frequency of occurrences of quality-related problems to highlight those that need the most attention.

Bars represent the individual values ordered by decreasing magnitude. A line represents the cumulative total. The left vertical axis is the frequency of occurrence or some other unit of measurement (such as cost or time). The right vertical axis is the cumulative frequency expressed as a percentage or total of the unit of measurement (such as total cost, total time).

Note: In user-interface controls and documentation, it is typical to refer to the count/frequency for usability and clarity rather than more generic terms such as measure/unit of measurement.

Related tasks

Creating a Pareto chart

Creating comparative Pareto charts

Creating a Pareto chart

Plot a Pareto chart to identify the most frequently occurring quality-related problems.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Pareto, and then click Single Pareto Chart.
The analysis task pane opens.

3. In the Variable drop-down list, select the categorical variable.

4. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

5. In the Primary axis drop-down list, select:

Option

Description

Count

Show the frequency of occurrence or unit of measurement.

Percent

Show the percentage of the total frequency or a total of the unit of measurement.

6. Optional: To force the primary and secondary axis so that 1 unit on the primary axis is equal to the percentage of the total on the secondary axis, select the Lock axes scales check box.

7. Optional: To label the bars, select the Label bars check box.

8. Optional: To label the line, select the Label cumulative line check box

9. Click Calculate.

Related concepts

Pareto chart

Related tasks

Reordering Pareto chart bars

Merging Pareto chart categories

Varying Pareto chart bar colors

Related reference

Dataset layout

Creating comparative Pareto charts

Plot a series of Pareto charts to identify the most frequently occurring quality-related problems stratified by up to 2 factors.

1. Select a cell in the dataset.

2. On the Analyse-it ribbon tab, in the Statistical Analyses group, click Pareto, and then click Comparative Pareto Chart.
The analysis task pane opens.

3. In the Variable drop-down list, select the categorical variable.

4. If the data are in frequency form, in the Frequency drop-down list, select the frequency count variable.

5. In either or both the Factor (column) and Factor (row) drop-down lists, select the categorical stratification variables.

6. In the Primary axis drop-down list, select:

Option

Description

Count

Show the frequency of occurrence or unit of measurement.

Percent

Show the percentage of the total frequency or a total of the unit of measurement.

7. Optional: To force the primary and secondary axis so that 1 unit on the primary axis is equal to the percentage of the total on the secondary axis, select the Lock axes scales check box.

8. Optional: To label the bars, select the Label bars check box.

9. Optional: To label the line, select the Label cumulative line check box

10. Optional: In the Layout drop-down list, select:

Option

Description

Flow

Layout the plots across the width of the page, before flowing onto a new row.

Matrix

Layout the plots in a matrix as defined by the row and column factors.

11. Click Calculate.

Related concepts

Pareto chart

Related tasks

Sorting bars on comparative Pareto charts

Reordering Pareto chart bars

Merging Pareto chart categories

Varying Pareto chart bar colors

Related reference

Dataset layout

Merging Pareto chart categories

Merge categories to reduce clutter and see the vital few rather than the trivial many.

You must have already completed either of the tasks:

- Creating a Pareto chart

- Creating comparative Pareto charts

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Pareto group, click Combine.
The analysis task pane, the Categories panel opens.

3. To merge all the categories with a frequency less than a threshold:

1. In the Combine group, select the Threshold option.

2. In the Less than value / % edit box, type the frequency count or relative frequency % threshold.

3. In the Name edit box, type the name of the newly merged category.

4. To merge a fixed number of smallest categories:

1. In the Combine group, select the Last option.

2. In the Categories / % edit box, type the number of categories to merge, or the % of categories.

3. In the Name edit box, type the name of the newly merged category.

5. To merge specific categories:

1. In the Combine group, select the Custom option.

2. In the Categories grid, click to select the first category, and then hold down SHIFT and click to select the last category to make a contiguous selection, or hold down CTRL and click to select additional categories to make a discontiguous selection.

3. Click Merge.

4. In the Categories grid, under the Name column, alongside the category in the edit box, type the name of the newly merged category.

6. Click Recalculate.

Varying Pareto chart bar colors

Color bars to highlight the importance of the vital few.

You must have already completed either of the tasks:

- Creating a Pareto chart

- Creating comparative Pareto charts

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Pareto group, click Vary Colors.
Each bar is varied in color using the color assigned to the categories of the variable in the dataset.

3. To color the bars to highlight some other attribute such as importance:

1. Select the Assign Colors check box.

2. In the Categories grid, under the Color column alongside the category, in the drop-down list, select the color. If you do not assign a color, the bar is colored gray.

3. Repeat step 3.b for each category.

4. Click Recalculate.

Related tasks

Assigning colors/symbols to categories

Reordering Pareto chart bars

Reorder bars to put a specific category at the end (for example, Miscellaneous).

You must have already completed either of the tasks:

- Creating a Pareto chart

- Creating comparative Pareto charts

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Pareto group, click Reorder.
The analysis task pane Categories panel opens.

3. Select the Keep specific category at end check box.

4. In the drop-down list, select the category to place at the end.

5. Click Recalculate.

Sorting bars on comparative Pareto charts

Sort the bars into the same order to make comparisons easier.

You must have already completed either of the tasks:

- Creating a Pareto chart

- Creating comparative Pareto charts

1. Activate the analysis report worksheet.

2. On the Analyse-it ribbon tab, in the Pareto group, click Reorder.
The analysis task pane Categories panel opens.

3. Select the Sort all by same key check box.

4. In the drop-down list(s), select the plot to use as the sort order.

5. Click Recalculate.

Study design

Pareto analysis study requirements and dataset layout.

Requirements

- A categorical variable of failures.

- 1 or 2 factor variables.

Dataset layout

Use a column for the variable (Failute), an optional column for the factor variable (Operator); each row has the values of the variables for a failure (ID).

ID (optional)

Operator (optional)

Failure

1

SNH

Colorimeter drift

2

SNH

Miscellaneous

3

JDH

Electrode failure

4

GMH

Deformed tubing

5

SNH

Reagents

6

SNH

Light failure

7

JDH

Deformed tubing

8

SNH

Colorimeter drift

9

JDH

Colorimeter drift

10

GMH

Deformed tubing

…

…

…

Frequency form dataset layout

Use a column for the variable (Failute), an optional column for the factor variable (Operator) and a column for the number of cases (Frequency); each row has the values of the variables and the frequency count.

Failure

Operator (optional)

Frequency

Colorimeter drift

SNH

14

Colorimeter drift

JDH

15

Colorimeter drift

GMH

22

Deformed tubing

SNH

4

Deformed tubing

JDH

5

Deformed tubing

GMH

5

Electrode failure

SNH

1

Electrode failure

JDH

2

Electrode failure

GMH

3

…

…

…

End-User License Agreement (EULA)

THIS END-USER LICENSE AGREEMENT (“EULA”) IS A LEGAL AGREEMENT BETWEEN YOU (EITHER AN INDIVIDUAL OR A SINGLE ENTITY) AND ANALYSE-IT SOFTWARE LIMITED FOR THE SOFTWARE PRODUCT ANALYSE-IT FOR MICROSOFT EXCEL, WHICH INCLUDES COMPUTER SOFTWARE, PRINTED MATERIALS, AND ELECTRONIC DOCUMENTATION (“SOFTWARE”). BY INSTALLING, COPYING, OR OTHERWISE USING THE SOFTWARE, YOU AGREE TO BE BOUND BY THE TERMS OF THIS EULA. IF YOU DO NOT AGREE TO THE TERMS OF THIS EULA, DO NOT INSTALL OR USE THE SOFTWARE, AND DESTROY ALL COPIES.

The SOFTWARE is protected by copyright and international copyright treaties, as well as other intellectual property laws and treaties. The SOFTWARE is licensed, not sold.

GRANT OF LICENSE

ANALYSE-IT SOFTWARE LIMITED grants you a non-exclusive license to use the SOFTWARE subject to restrictions:

A time-limited license permits use of the SOFTWARE during the specified time period. After the time period expires you must either (a) destroy all copies of the SOFTWARE, or (b) purchase a license to use the SOFTWARE.

A fixed per-user user license permits a fixed number of users to install and use the software on their computer, and optionally on their laptop or home computer. The SOFTWARE may be installed and used only by the number of users as you have fixed per-user licenses for.

A floating concurrent-user license permits an unlimited number of users to install the software on their computer or laptop computer, but limits the number of users of the software at any time. The SOFTWARE may be used concurrently by as many users as you have floating concurrent-user licenses for. Use of the SOFTWARE requires an active Internet connection to our license server to control concurrent usage.

OTHER RIGHTS AND LIMITATIONS

You may not reverse engineer, decompile, or disassemble the SOFTWARE, except and only to the extent that such activity is expressly permitted by applicable law notwithstanding this limitation.

You may not attempt to bypass any license control restrictions imposed by the SOFTWARE.

You may not rent, lease, or loan the SOFTWARE.

You agree to hold the SOFTWARE in confidence and to take reasonable steps to prevent unauthorised copying.

You may permanently transfer all your rights under this EULA, provided you retain no copies, you transfer all of the SOFTWARE, and the recipient agrees to the terms of this EULA. If the SOFTWARE is an upgrade, any transfer must include all prior versions of the SOFTWARE.

LICENSE VIOLATION

We collect information about your computer, including a unique computer identifier to determine the number of installations of the software, and also collect information about your version of Windows, Excel, your IP address, and your name / company / e-mail address. This information is held in the strictest confidence and is used to inform you of updates and also to detect, prevent and prosecute violation of the license terms and conditions.

DUAL VERSIONS

You may receive multiple versions of the SOFTWARE for different versions of Microsoft Excel. Regardless you may only install or use the SOFTWARE on a single computer system located at your site. You may not use or install the other versions on another computer, nor loan, rent, lease or otherwise transfer them except as part of a permanent transfer as provided above.

UPGRADES

If the SOFTWARE is labelled as an upgrade, you must be properly licensed to use a product identified by ANALYSE-IT SOFTWARE LIMITED as being eligible for the upgrade. SOFTWARE labelled as an upgrade replaces and/or supplements the product that formed your eligibility for the upgrade. You may use the resulting updated SOFTWARE only in accordance with the terms of this EULA.

COPYRIGHT

All title and copyrights in and to the SOFTWARE, the accompanying printed materials, and any copies of the SOFTWARE, are owned by ANALYSE-IT SOFTWARE LIMITED. The SOFTWARE is protected by copyright laws and international treaty provisions. Therefore, you must treat the SOFTWARE like any other copyrighted material except that you may either (a) make one copy of the SOFTWARE solely for backup or archival purposes, or (b) install the SOFTWARE in accordance with the Grant of License provided you keep the original solely for backup or archival purposes. You may not copy the printed materials accompanying the SOFTWARE.

LIMITED WARRANTY

ANALYSE-IT SOFTWARE LIMITED makes no warranty, expressed or implied, about the merchantability or fitness for a particular purpose, of the SOFTWARE during the evaluation period. You install it, and evaluate it, entirely at your own risk.

Only when a license has been purchased does ANALYSE-IT SOFTWARE LIMITED warrant that the SOFTWARE will perform substantially in accordance with the accompanying documentation for a period of ninety (90) days from the date of receipt. Any implied warranties on the SOFTWARE are limited to ninety (90) days, or the shortest period permitted by applicable law, whichever is greater.

To the maximum extent permitted by the applicable law, ANALYSE-IT SOFTWARE LIMITED disclaims all other warranties, either express or implied, including but not limited to implied warranties of merchantability and fitness for a particular purpose, with respect to the SOFTWARE, the accompanying product manual(s) and written material. The Limited Warranty contained herein gives you specific legal rights.

CUSTOMER REMEDIES

ANALYSE-IT SOFTWARE LIMITED’s entire liability and your exclusive remedy shall be, at ANALYSE-IT SOFTWARE LIMITED’s option, either (a) return of the price paid or (b) repair or replacement of the SOFTWARE that does not meet ANALYSE-IT SOFTWARE LIMITED’s Limited Warranty and which is returned to ANALYSE-IT SOFTWARE LIMITED with a copy of the receipt. This Limited Warranty is void if failure of the SOFTWARE has resulted from an accident, abuse, or misapplication. Any replacement SOFTWARE will be warranted for the remainder of the original warranty period or thirty (30) days, whichever is longer.

LIMITED LIABILITY

To the maximum extent permitted by applicable law, ANALYSE-IT SOFTWARE LIMITED shall not be liable for any other damages whatsoever (including, without limitation, damages for loss of business profits, business interruption, business information, or other pecuniary loss) arising out of the use or inability to use this product, even if ANALYSE-IT SOFTWARE LIMITED has been advised of the possibility of such damages. In any case, ANALYSE-IT SOFTWARE LIMITED’s entire liability under any provision of this agreement shall be limited to the price paid for the SOFTWARE.

TERMINATION

Without prejudice to any other rights, ANALYSE-IT SOFTWARE LIMITED may terminate this EULA if you fail to comply with the terms and conditions herein. In such event, you must destroy all copies of the SOFTWARE.

GOVERNING LAW

This EULA is governed by the laws of England, UK.

Bibliography

List of references to articles and books.

Altman, D. G., & Bland, J. M. (1983). Measurement in medicine: the analysis of method comparison studies. The statistician, 307-317.

Bablok, W., Passing, H., Bender, R., & Schneider, B. (1988). A general regression procedure for method transformation. Application of linear regression procedures for method comparison studies in clinical chemistry, Part III. Clinical Chemistry and Laboratory Medicine, 26(11), 783-790.

Bangdiwala, S. I., & Shankar, V. (2013). The agreement chart. BMC medical research methodology, 13(1), 97.

Bland, J., & Altman, D. (1986). Statistical methods for assessing agreement between two methods of clinical measurement. The Lancet, 327(8476), 307-310.

Bland, J. M., & Altman, D. G. (1995). Comparing methods of measurement: why plotting difference against standard method is misleading. The Lancet, 346(8982), 1085-1087.

Bland, J. M., & Altman, D. G. (1999). Measuring agreement in method comparison studies. Statistical methods in medical research, 8(2), 135-160.

Burdick, R. K. & Graybill, F. A. (1992). Confidence intervals on variance components. CRC Press.

Burr, I. W. (1967). The effect of non-normality on constants for Xbar and R charts. Industrial Quality Control, 23(11), 563-569.

Carstensen, B. (2011). Comparing clinical measurement methods: a practical guide. John Wiley & Sons.

CLSI. (2014a). Evaluation of Precision Performance of Quantitative Measurement Methods (EP5-A3). Clinical and Laboratory Standards Institute.

CLSI. (2003). Evaluation of the Linearity of Quantitative Measurement Procedures (EP6-A). Clinical and Laboratory Standards Institute.

CLSI. (2020). Evaluation of the Linearity of Quantitative Measurement Procedures (EP6-Ed2). Clinical and Laboratory Standards Institute.

CLSI. (2002). Method Comparison and Bias Estimation Using Patient Samples (EP9-A2). Clinical and Laboratory Standards Institute.

CLSI. (2014b). User Verification of Performance for Precision and Trueness (EP15-A3). Clinical and Laboratory Standards Institute.

CLSI. (2012). Evaluation of Detection Capability for Clinical Laboratory Measurement Procedures (EP17-A2). Clinical and Laboratory Standards Institute.

CLSI. (2013). Estimation of Total Analytical Error for Clinical Laboratory Methods (EP21-A). Clinical and Laboratory Standards Institute.

CLSI. (2010). Defining, Establishing, and Verifying Reference Intervals in the Clinical Laboratory (EP28-A3C, formerly C28-A3C). Clinical and Laboratory Standards Institute.

Cicchetti, D. V., & Feinstein, A. R. (1990). High agreement but low kappa: II. Resolving the paradoxes. Journal of clinical epidemiology, 43(6), 551-558.

Cornbleet, P. J., & Gochman, N. (1979). Incorrect least-squares regression coefficients in method-comparison analysis. Clinical Chemistry, 25(3), 432-438.

Currie, L. A. (1999). Detection and quantification limits: origins and historical overview. Analytica Chimica Acta, 391(2), 127-134.

DeLong, E. R., DeLong, D. M., & Clarke-Pearson, D. L. (1988). Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 837-845.

Gabriel, K. R. (1971). The biplot graphic display of matrices with application to principal component analysis. Biometrika, 58(3), 453-467.

Emancipator, K., & Kroll, M. H. (1993). A quantitative measure of nonlinearity. Clinical chemistry, 39(5), 766-772.

Feinstein, A. R., & Cicchetti, D. V. (1990). High agreement but low kappa: I. The problems of two paradoxes. Journal of clinical epidemiology, 43(6), 543-549.

Gower, J. C., Lubbe, S. G., & Le Roux, N. J. (2011). Understanding Biplots. John Wiley & Sons.

Harrell, F. E., & Davis, C. E. (1982). A new distribution-free quantile estimator. Biometrika, 69(3), 635-640.

Hanley, J. A., & McNeil, B. J. (1982). The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology, 143(1), 29-36.

Hanley, J. A., & McNeil, B. J. (1983). A method of comparing the areas under receiver operating characteristic curves derived from the same cases. Radiology, 148(3), 839-843.

Hsieh, E., & Liu, J. P. (2008). On statistical evaluation of the linearity in assay validation. Journal of Biopharmaceutical Statistics, 18(4), 677-690.

Hsieh, E., Hsiao, C. F., & Liu, J. P. (2009). Statistical methods for evaluating the linearity in assay validation. Journal of Chemometrics, 23(1), 56-63.

Hirji, K. F. (2005). Exact analysis of discrete data. CRC Press.

Horn, P. S. (1990). Robust quantile estimators for skewed populations. Biometrika, 77(3), 631-636.

Horn, P. S., Pesce, A. J., & Copeland, B. E. (1998). A robust approach to reference interval estimation and evaluation. Clinical Chemistry, 44(3), 622-631.

Horn, P. S., Pesce, A. J., & Copeland, B. E. (1999). Reference interval computation using robust vs. parametric and nonparametric analyses. Clinical Chemistry, 45(12), 2284-2285.

Horn, P. S., & Pesce, A. J. (2005). Reference intervals: a user's guide. American Association for Clinical Chemistry.

Hsu, J. C. (1996). Multiple Comparisons: Theory and Methods. CRC Press.

ISO. (1994). ISO 5725: Accuracy (Trueness and Precision) of Measurement Methods and Results. International Organization for Standardization.

Jolliffe, I. (2002). Principal component analysis. Springer-Verlag, New York, Inc.

Kroll, M. H., & Emancipator, K. (1993). A theoretical evaluation of linearity. Clinical chemistry, 39(3), 405-413.

Krouwer, J. S. (2008). Why Bland–Altman plots should use X, not (Y+ X)/2 when X is a reference method. Statistics in Medicine, 27(5), 778-780.

Krouwer, J. S., & Monti, K. L. (1995). A simple, graphical method to evaluate laboratory assays. European journal of clinical chemistry and clinical biochemistry, 33(8), 525-528.

Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. biometrics, 159-174.

Linnet, K. (1993). Evaluation of regression procedures for methods comparison studies. Clinical Chemistry, 39, 424-424.

Linnet, K. (1990). Estimation of the linear relationship between the measurements of two methods with proportional errors. Statistics in Medicine, 9(12), 1463-1473.

Linnet, K. (1998). Performance of Deming regression analysis in case of misspecified analytical error ratio in method comparison studies. Clinical chemistry, 44(5), 1024-1031.

Linnet, K. (2000). Nonparametric estimation of reference intervals by simple and bootstrap-based procedures. Clinical Chemistry, 46(6), 867-869.

Krouwer, J. S. (2002). Setting performance goals and evaluating total analytical error for diagnostic assays. Clinical Chemistry, 48(6), 919-927.

Linnet, K., & Kondratovich, M. (2004). Partly nonparametric approach for determining the limit of detection. Clinical chemistry, 50(4), 732-740.

Liu, A., & Bandos, A. I. (2012). Statistical evaluation of diagnostic performance: topics in ROC analysis. CRC Press.

Monti, K. L. (1995). Folded empirical distribution function curves—mountain plots. The American Statistician, 49(4), 342-345.

Montgomery, D. C. (2012). Introduction to statistical quality control. John Wiley & Sons.

Nelson, L. S. (1984). Technical Aids: The Shewhart Control Chart -Tests for Special Causes. Journal of quality technology, 16(4).

Passing, H., & Bablok, W. (1983). A new biometrical procedure for testing the equality of measurements from two different analytical methods. Application of linear regression procedures for method comparison studies in clinical chemistry, Part I. Clinical Chemistry and Laboratory Medicine, 21(11), 709-720.

Passing, H., & Bablok, W. (1984). Comparison of Several Regression Procedures for Method Comparison Studies and Determination of Sample Sizes Application of linear regression procedures for method comparison studies in Clinical Chemistry, Part II. Clinical Chemistry and Laboratory Medicine, 22(6), 431-445.

Pepe, M. S. (2003). The statistical evaluation of medical tests for classification and prediction. Oxford University Press.

Pollock, M. A., Jefferson, S. G., Kane, J. W., Lomax, K., MacKinnon, G., & Winnard, C. B. (1992). Method comparison - a different approach. Annals of clinical biochemistry, 29, 556-560.

Rigdon, S. E., Cruthis, E. N., & Champ, C. W. (1994). Design strategies for individuals and moving range control charts. Journal of Quality Technology, 26(4), 274-287.

Ryan, T. P. (2011). Statistical methods for quality improvement. John Wiley & Sons.

Sadler, W. A., & Smith, M. H. (1986). A reliable method of estimating the variance function in immunoassay. Computational Statistics & Data Analysis, 3, 227-239.

Sadler, W. A., Smith, M. H., & Legge, H. M. (1988). A method for direct estimation of imprecision profiles, with reference to immunoassay data. Clinical chemistry, 34(6), 1058-1061.

Sadler, W. A., (2016). Using the variance function to estimate limit of blank, limit of detection and their confidence intervals. Ann Clin Biochem, 53, 141-149.

Strike, P. W. (2014). Statistical methods in laboratory medicine. Butterworth-Heinemann.

N. I. S. T. (2012). e-Handbook of Statistical Methods.

Sahai, H., & Ojeda, M. M. (2004). Analysis of Variance for Random Models, Volume 1: Balanced Data. Springer Science & Business Media.

Sahai, H., & Ojeda, M. M. (2004). Analysis of Variance for Random Models, Volume 2: Unbalanced Data. Springer Science & Business Media.

Solberg, H. E. (1987). Approved recommendation on the theory of reference values. Part 5. Statistical treatment of collected reference values. Determination of reference limits. Clinica Chimica Acta, 170(2), S13-S32.

Stöckl, D., Dewitte, K., & Thienpont, L. M. (1998). Validity of linear regression in method comparison studies: is it limited by the statistical model or the quality of the analytical input data?. Clinical Chemistry, 44(11), 2340-2346.

Western Electric Company. (1958). Statistical Quality Control Handbook. AT&T.

Xue J., Titterington D. M. (2010, unpublished). The p-folded Cumulative Distribution Function and the Mean Absolute Deviation from the p-quantile.

Zhou, X. H., Obuchowski, N. A., & McClish, D. K. (2011). Statistical methods in diagnostic medicine. Wiley-Blackwell.

Zweig, M. H., & Campbell, G. (1993). Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clinical Chemistry, 39(4), 561-577.