BayesWeb

Instructions

What this website does

This website allows you to perform exploratory Bayesian analyses of event rates observed in single-arm experiments. The site was designed to facilitate the exploration of potential safety issues associated with a medical product in early-phase clinical trials, but the methods are applicable to any problem where the parameter of interest is the probability of occurrence of some binary outcome.

How Bayesian analysis works

In a Bayesian analysis, information external to an experiment is used to form a prior probability distribution for the parameter of interest (in this case, a proportion). This prior distribution expresses the analyst's belief about where the parameter is likely to lie and how much confidence the analyst has about the value of the parameter before the experiment is conducted. The prior distribution can be based on historical data or expert opinion, or can be chosen to represent a state of relative ignorance about the parameter (a so-called non-informative prior).

The prior distribution is then combined with data from the experiment to form an updated posterior probability distribution for the parameter of interest. The posterior distribution represents the revised state of belief about where the parameter is likely to lie after taking into account both the prior information and the experimental data. The posterior distribution can then be explored to come to an understanding of the experimental data in the wider context represented by the prior distribution.

For more detailed information on Bayesian statistics, Wikipedia isn't a bad place to start.

How to use this site

The plots to the left show the current prior distribution (top), the likelihood function for the experimental data that have been input (middle) and the current posterior distribution (bottom).

The tabs at the top of the page provide access to the four main sections of this site. Under the Prior distribution tab, there are a number of ways to interact with the program to form a prior distribution, including providing guesses as to the most likely value for the parameter and the confidence you have in that value, providing historical data, or choosing a non-informative prior. Each menu item for forming a prior distribution contains its own instructions. Note that you may use more than one method for forming a prior distribution; in this case, the resulting prior distribution will represent a compromise between the distribution generated by each menu item individually.

The top of the prior distribution section contains a table showing interactive summary statistics about the currently specified prior distribution. These statistics can be used as a check on the information you've provided. For instance, if you specify a prior based on historical data but then find that the 95% credible interval is unrealistically wide or narrow, you can revise the information you used to form the prior to obtain more realistic results.

The Study data tab allows you to enter the number of subjects and number of events observed in your experiment. A table of summary statistics provides access to typical frequentist analyses of binary data, including exact binomial confidence intervals and hypothesis tests.

When a prior distribution has been specified and study data have been input, the Posterior distribution tab provides an interactive table of summary statistics concerning the posterior distribution. This table can be used to explore the posterior distribution in order to understand the updated state of belief about the event rate of interest. The summary tables from the prior distribution and study data sections are also repeated here for reference.

The Sensitivity analysis section allows you to explore how changes in the prior mode and sample size would affect the conclusions of your analysis.

Sensitivity analysis

You can use the sliders and boxes below to explore how different assumptions about the prior distribution would affect your conclusions. The prior mode is the single most likely value for the event rate before performing the experiment. The prior sample size refers to the effective sample size of the prior distribution (i.e., how many subjects worth of data the prior represents). Note that checking the box below will temporarily override your other choices for the prior distribution.

Sensitivity analysis mode

Prior mode:
Prior sample size:

Posterior distribution summary

Center	Spread	Credible interval	Probability that proportion is	Percentile	Beta parameters
					a	b

Current prior

Center	Spread	Credible interval	Probability that proportion is	Percentile	Beta parameters
					a	b

Determining the prior distribution

Use one or more of the following sections to help determine the prior distribution for the event rate. If more than one method is used to determine a prior, note that the final prior distribution will represent a compromise between each prior specified. We recommend this approach in general.

Specify prior by center and spread

One way to specify a prior distribution is to provide your best guess for the center of the prior and for its variability. The center can be specified as the mean, the mode or the median. (The mode is the single most likely value, and the median is the value such that there is a 50% probability that the event rate is greater than and 50% probability that the event rate is less than the median.) The variability of the distribution can be specified as the variance, the standard deviation, or as a credible interval. Credible intervals may be the most intuitive measure of variability. As an example, a prior 99% credible interval of (0.20, 0.50) means that you feel there is a 99% probability that the event rate falls between 20% and 50%. In many cases, this would be equivalent to saying that 20% - 50% was the range of plausible values.

Center	Spread

	,

Specify interval probabilities

Interval	Probability
0.00 - 0.20
0.20 - 0.40
0.40 - 0.60
0.60 - 0.80
0.80 - 1.00

Interval	Probability	Interval	Probability
0.00 - 0.10		0.50 - 0.60
0.10 - 0.20		0.60 - 0.70
0.20 - 0.30		0.70 - 0.80
0.30 - 0.40		0.80 - 0.90
0.40 - 0.50		0.90 - 1.00

Interval	Probability	Interval	Probability
0.00 - 0.05		0.50 - 0.55
0.05 - 0.10		0.55 - 0.60
0.10 - 0.15		0.60 - 0.65
0.15 - 0.20		0.65 - 0.70
0.20 - 0.25		0.70 - 0.75
0.25 - 0.30		0.75 - 0.80
0.30 - 0.35		0.80 - 0.85
0.35 - 0.40		0.85 - 0.90
0.40 - 0.45		0.90 - 0.95
0.45 - 0.50		0.95 - 1.00

Historical data

If you have pertinent historical data (from the literature, from prior studies, from case review, etc.), enter the total number of events and the total sample size here. You can also include a 'discounting' factor if you feel the prior information should be given less weight than current data.

Hypothetical subjects

For some people, imagining what might happen to hypothetical study subjects is a useful method for quantifying prior belief. You can answer each of the questions below about hypothetical subjects with a probability between 0 and 1 (or a percentage).

Prior mode and sample size

Sometimes providing a 'best guess' for an event rate and your confidence in the strength of that guess in the form of the number of hypothetical subjects you feel that guess is 'worth' is an intuitive way to specify a prior distribution. You can provide your best guess for the probability of an event (the mode) and your confidence in that guess in the form of "how many subjects worth of data" you feel your guess is based on here.

Noninformative or 'ignorance' priors

If you have no or completely ambiguous prior information, you can use a so-called non-informative prior to express the state of ignorance. The flat prior assumes equal prior probability that the event rate lies in any interval of a given size and is equivalent to two hypothetical prior subjects' worth of information. The Jeffreys prior has certain attractive mathematical properties and is equivalent to one hypothetical prior subject worth of information. The flat prior is generally more intuitive to non-statisticians and is the default.

Input beta parameters directly

You can provide explicit values for the parameters of the prior beta distribution here. This is recommended only for users who are familiar with Bayesian analysis for binomial parameters or in situations where a prior distribution has been previously established (e.g., in the literature).

Observed study data

Enter the data observed in your study here. This section may also be useful for exploring hypothetical experimental scenarios that would lead to specific conclusions along with a given prior distribution.

Data summary

The numbers presented in this table represent a standard 'frequentist' analysis of the study data.

Event rate	Confidence interval	p-value for test of H₀: p

Imagine that a single subject is going to be enrolled in a study. What would you say is the probability that this subject will experience an event?	Now imagine that the first subject did experience an event. What would you now say is the probability that the second subject will also experience an event?
Probability of an event:	Probability of a second event:

Prior distribution

Likelihood

Posterior distribution