Score Normalization in Beamtime Allocation
Note: This procedure affects only heavily oversubscribed beamlines with cutoff scores; protein crystallography beamlines are not affected.
The General Sciences Proposal Study Panel (PSP) meets every six months to evaluate general user proposals and to advise the ALS on the procedures by which beamtime is assigned. Outside experts and at least two members of the PSP review the proposals and rate each one on a numerical scale of 1 (highest) to 5 (lowest). The scores are then averaged and used as the primary factor in allocating beamtime for general users. Most proposals remain active for four six-month cycles.
In the past, beamtime allocation was based on a proposal's score and the cutoff score for a particular beamline. If a proposal had a score better than the cutoff score, investigators were allocated beamtime for the first cycle of the proposal. If a proposal scored below the assigned cutoff score, beamtime was not granted. The consequence of this process was that proposals scoring just above the cutoff could be assigned beamtime for all four cycles (if the proposal was maintained "active" for the entire time), while proposals scoring just below the cutoff would not be assigned any beamtime over the same period. However, because the error in determining proposal scores can be larger than the differences between some average proposal scores, it was determined that a more equitable manner of allocating beamtime for similar proposals near the cutoff score should be considered.
The General Sciences PSP has recommended the practice of "score normalization" for proposals with similar scores near the cutoff score as a method of automating the process and of achieving a more equitable allocation of beamtime. The PSP, Users' Executive Committee (UEC), and Scientific Advisory Committee (SAC) have since approved a procedure by which a delta (D)--in this case 0.1--is added to the scores of those proposals that received beamtime in the cycle preceding the one for which beamtime allocation is under consideration. In the example below, scores for 18 proposals are scored from 1.60 (highest) to 2.45 (lowest), where the cutoff score is 2.0, and D=0.1.
Score Normalization Model
In this example, each six-month scheduling cycle can accommodate nine proposals for beamtime from general users. In the first six-month cycle (1), proposals 1 - 9 each receive some beamtime (shaded pink), while proposals 10 - 18 receive none (shaded blue). For the next six-month cycle, all proposals that were allocated beamtime in the previous cycle have their scores adjusted by D = 0.1. Scores for proposals that did not not receive beamtime in the previous cycle are not changed. As a result, in the second running cycle (2), proposals 1 - 8, and 10, (shaded pink), receive beamtime, while proposals 9 and 11 - 18, do not (shaded blue). The process continues for each cycle with D added only to the scores of the proposals that received beamtime.
Under the score normalization procedure described above, user groups with similarly scored proposals near the cutoff for a specific beamline will receive 100, 75, 50, 25, or 0% of their requested beamtime over a two-year period. It should be noted however, that inclusion of new, highly ranked proposals during this time could alter beamtime distribution. Proposal score nomalization has been successfully used at the ALS in assigning beamtime for the general sciences since May 2001.
Additional Information
For information about beamlines available to general users,
contact:
User Services Group Leader
Advanced Light Source, MS 6-2100
Berkeley Lab
Berkeley, CA 94720
Tel: (510) 486-7727
Fax: (510) 486-4773