Daily Run10 Journal

Jan. 8 The past day was another difficult struggle to get beam
            lifetimes under better control.
            Yesterday morning was given over for experimental
            access. Problems with the accesses forced a sweep
            of the experimental areas and there occured a
            problem with a switch in the access controls.
            So the 2 hour access stretched out to over 4
            hours without any beam activity.
            Before beam work could begin, a problem with the
            RHIC BPM system was enountered. This took another
            hour to sort out. Beam was not injected and ramped
            until after 4pm.
            The evening was spent working on commissioning
            a new ramp, which had modifications in it to improve
            beam lifetime at store (otherwise the ramp was no
            different than for physics stores). This work continued
            until early this morning. There remain some issues
            related to this work, most significantly rebucketing
            was left off for the most recent store.
Jan. 7 We are currently focused on improving store lifetimes, which
            are now limiting luminosity. The store put in early
            this morning was 84x84 with 1.1e9, but had lower ZDC
            rates than the previous physics store that had fewer
            bunches and lower intensity/bunch. This was all due
            to poor lifetimes.
            Overnight was for APEX. Four areas were studied: IR optics,
            orbit response matrix measurements, orbit at PHENIX,
            and IR corrections. These all went very well.
            Yesterday afternoon we were off due to two failures:
            sextupole tripped off (took time to recover) and
            the injection kickers started firing while ramping.
            The second problem was caused by a configuration error.
Jan. 6 Yesterday we went into an intensive 11 hour development period.
            This was admittedly not well planned out, but
            it proved to be highly productive. The main original
            goal of this development was to measure the chromaticity
            (first and second order) and dynamic aperture with a
            6x6 bunch ramp. What was found was chromaticities (first
            order) were far from where they should have been in a
            number of places. In particular the vertical chromaticity
            at Yellow store was large and negative, while it should
            be small and positive. So we chose to correct these
            In addition, the LLRF group was ready to put in some
            fixes to problems encountered during commissioning.
            A major problem has been in the injection cogging.
            We have not been able to inject more than one bunch
            per AGS cycle. This leaves beam at injection for too
            long, causing significant debunching from IBS. They
            were able to fix this partially, so we can now inject
            3 out of the 4 bunches in the AGS in one cycle.
            Finally, work was done to improve the rebucketing,
            which is now working much better and with higher voltages.
            We changed the store lengths to be 4 hours starting last
            night. Previously we had set them to 6 hours, to give more
            setup time to the experiments. Now we want to ramp up
            intensity as fast as possible. Each new store will be
            incrementally higher intensity than the previous one.
Jan. 5 We continue to work on increasing intensity and run.
            Yesterday an access was made for Stochastic cooling
            to work on improving the signals from their pickups.
            We also worked on improving the rebucketing, adding
            two common cavities to achieve higher voltage.
            We lost one store last night due to a problem in the
            LLRF, causing a significant amount of beam to debunch.
            The last store put up was with 68x68 bunches with
            1e9 ions/bunch.
Jan. 4 We have started progressively raising intensity each store.
            The most recent store (11365, 1/4/10 at 6:14am) was
            with 68x68 bunches with 0.9e9 ions/bunch.
            Here is how the weekend stacked up. ZDC's and Intensity over the weekend.
Jan. 3 The beam loss that was preventing our going to higher
            intensity was cured today. Over the next few days
            we will progressively push the intensity up.
            Problems encountered today were system level problems.
            Early in the morning there was problem with injecting
            beam, associated with the bunch suppression configuration.
            Later in the morning there was a quench link interlock.
            This was followed by an independent controls problem with
            a power supply FEC. After clearing these problems a new
            fill was put up at 3 pm. This fill was with modest
            intensity, but with corrections put in late in the ramp
            cycle, the losses that were holding us from going to high
            intensity had dropped significantly.
Jan. 2 Overnight we ran with two modest stores. Each was with 56x56
            bunches with about 0.85e9 ions/bunch.
            Today, as we tried to raise intensity, either
            by increasing the bunch intensity or the number
            of bunches, we are inhibited by losses in the
            STAR triplets during the beta-squeeze.
            Efforts are now going into reducing losses
            and raising intensity.
Jan. 1 Last night a 56x56 store with 1e9 ions/bunch was put up,
            with rebucketing working.
            We are now in a mode where we will be putting up
            stores for physics, as much as possible. At the
            same time we will be increasing intensity and
            the number of bunches.
            During the days next week we will be working on
            stochastic cooling commissioning.
Dec. 31 Most of the day was spent working on commissioning the
            new LLRF for the storage system (200 MHz) RF cavities.
            The cavities were phased in and HL control setup, but
            the final step of getting beam rebucketed from the
            28 MHz RF into the 200 MHz RF needs to be done today.
            Venier scans were performed at both STAR and PHENIX
            yesterday. ZDC cross section's were measured to be the
            same as run Run7.
            At 1:30 am we got a 56x56 bunch store up with ~0.6e9
            ions/bunch, for experiment setup.
Dec. 30 Some progress was made in the last day, although a number of
            failures slowed us down. Yesterday morning we were
            kept off due to a bad UPS on the QPS in 1010A. After
            that was fixed we were kept off due to lead flow interlocks.
            Late in the afternoon we were able to run again and tested
            ramping with higher intensity (0.9 to 1.1e9 ions/bunch).
            Three such ramps were attempted before a LLRF problem
            kept us off for the rest of the evening.
            At 4 am, this morning, we got a 56x56 bunch store up
            with ~0.7e9 ions/bunch, for experiment setup.
Dec. 29 Yesterday was spent on experiment setup using a 56x56 bunch store
            and a first look at stochastic cooling commissioning.
            In the evening work continued on LLRF commissioning
            and beam development (increasing intensity).
            This morning a 61x61 bunch store was established
            for experiment setup.
Dec. 28 Last night we put in our first 56x56 bunch store, for experiments
            to begin setup for physics. Unfortunately is was aborted by
            an abort kicker prefire. Experiments have seen beam to begin
            setup, though, over the weekend.
            Over the weekend progress was made in the new LLRF setup.
            Ring to Ring synchronization was commissioned as well as
            store cogging and timing. Both rings have Landau cavities on.
            Still to be done is to setup rebucketing in the storage cavities.
            We are now ramping up to physics, focusing on store development.
Dec. 26 Work continued over the holiday to improve efficiencies on
            the ramp and at store.
            Yesterday, for the first time in Run 10, beams were put
            into collisions at both STAR and PHENIX (Merry Christmas!).
            Before we can work on physics stores, the LLRF needs to
            get ring to ring synchronization and store cogging working.
            That is being worked on today.
Dec. 24 Tuning up of the ramp efficiencies continued today.
            Overnight we have brought beam to store energies in both rings.
            Ramp efficieny in Blue is close to 100%, Yellow efficiency is poor.
            Transition crossings were worked on yesterday and fixed in both
            rings. LLRF attempted to get the Ring to Ring synchronization
            working. This needs more work, so we have no collisions yet.
            Currently we are focusing on Yellow ramp efficiency.
Dec. 23 Ramp development in both Blue and Yellow continued.
            With the focus on tuning up the transition crossing.
            With both rings running synchronously, Blue beam
            gets to store, although with only about 75% ramp
            efficiency. Most beam loss is just after transition.
            For Yellow, most of the beam is lost at transition.
            Today more work needs to be done with LLRF and ramp efficiency.
Dec. 22 Development of beam in Yellow has begun. LLRF setup
            was completed around 2 am this morning.
            Currently Yellow is being tuned to get beam to store.
            Blue has developed a problem, since we started
            ramping the two rings together. Now beam dies
            before getting to store. Since the problem is
            associated with ramping together, we are investigating
            possible causes under that condition.
Dec. 21 Newsday reported 26.3 inches of snow - at BNL - when all was said and done.
            Beam development got going last night.
            Yellow is now available for ramp development.
            Blue didn't reproduce after coming on again,
            so effort is going into working on both Blue and Yellow today.
            Schedule-wise, we are now a few days behind,
            but still pushing to have collisions by Tues. or Wed., this week.
Dec. 20 Early this morning the Blizzard of '09 finally shut us down.
            According to the National Weather Service, BNL measured a record
            24.9 inches of snowfall as of 7 am this morning.
            Beam development in RHIC continued until 3 am, when a cooling
            tower for the Booster RF went down, shutting down the Booster.
            The rest of today was spent getting roads cleared so we
            could re-start operations.
            Blue is in good shape, with beam accelerated up to store.
            Work began last night on increasing intensity in Blue.
            Today Yellow became available, so beam development will
            begin tonight in Yellow.
Dec. 18 Blue ramp development continued overnight.
            A problem was encountered the night before, in which
            the set tunes needed to be near the integer, causing
            the on-line model to crash. The actual tunes of (31.23, 31.216)
            were achieved by setting the tunes down near 31.05. It is still
            being investigated why the model is not getting them right, but
            a strong likelihood is new transfer functions that were loaded
            into the database over the summer shutdown (to match magnetic
            measurements). We reverted back to the old database yesterday
            and developed a new ramp, with all the same features. Overnight
            this ramp was developed and by this morning beam in Blue is
            up to store with >90% ramp efficiency.
            Yellow remains off due to a problem with the Q7 Quench Protection
Dec. 17 Blue ramp development continued overnight, after being off all day
            for AGS Motor Generator repairs.
            The Blue ramp efficiency is improving, up to about 90%.
            Last night the PS group ran Yellow up to Store current levels.
            Today they need to test on the normal ramp for physics.
Dec. 16 Beam in Blue was accelerated to Store last night.
            PS group continued work on Yellow MMPS ramp development.
            Ramp development in Blue was expedited by using
            tune and coupling feedback for every ramp attempt and
            through the use of orbit feedback, which allowed correcting
            the orbit through transition in one ramp cycle.
            With a smaller beta-star, it is more difficult to correct
            the orbit during the beta-squeeze, so these systems helped
            speed up the process significantly.
            Michiko Minty put together a nice summary of the 1st 24 ramps in Blue.
            Blue ramp development summary.
Dec. 15 Ramp development for Blue began last night.
            Acceleration and the transition phase jump work well.
            LLRF setup for Blue is mostly complete, although they
            are still working on cogging.
            Yellow MMPS setup progressed today.
Dec. 14 Both Blue and Yellow are now setup at injection with RF capture.
            The MMPS for Blue is almost ready for ramp development.
            Yellow MMPS will be setup today for ramping.
            Tonight we expect to start ramp development with exp. magnets on.
Dec. 13 This afternoon we finally got beam circulating in Yellow.
            The problem turned out to be a quadrupole in the wrong polarity.
            Now we have begun to work on LLRF commissioning in Yellow.
            Also, last night, the MMPS group made progress in Blue, and
            we can now use Au101 again. We will begin working on ramp
            development in Blue tomorrow morning.
Dec. 12 Last night beam was injected into Yellow for the first time.
            Once synchro was set and the kickers timed in it became
            clear there was an obstruction in the area of 12 O'Clock.
            At most 3 turns of beam would survive.
            Today it was found the stochastic cooling pick-up at 12 O'Clock
            was not retracted. It would appear it was obstructing the beam.
            This evening, beam will be injected again and apertures around
            12 O'Clock and the Yellow cold snakes will be probed.
            LLRF commissioning for Yellow will be worked on tonight.
            For Blue, the PS group is still working on getting the Ramp
            to work. We have switched to using the Au100 ramp (which has
            the transition jumps at the same time for both rings) as oppposed
            to the Au101 ramp (which had them at different times).
            A distortion in the lattice around transition, to enable Blue
            and Yellow to jump transition at different times, was causing
            difficulties in tuning up the MMPS for that ramp. The Au100
            ramp doesn't have those distortions.
            So far they have almost gotten the current up to store levels, but
            more tuning is required.
Dec. 11 Work on Blue and Yellow MMPS continues.
            A PS failure yesterday took most of the day to fix
            (1st spare didn't work).
            Yellow could not run because of a problem in the Yellow valve box.
            This will be repaired today.
            One attempt to accelerate was made over night, but the ramp failed.
            Some beam was accelerated, though, so it was a useful first ramp.
            Blue injection is in good shape. Instrumentation is working well.
            All other systems are ready for ramp development.
Dec. 10 Worked on preparing Blue MMPS to ramp to store.
            Almost ready. Will finish today.
            LLRF Commissioning is coming along well.
            Not much beam development yesterday, but good thinking time
            (when not sleeping), so some improvements were still made
            (especially in "understanding").
            Some beam work was done last night, at injection.
            Blue injection optics are better understood.
            Instrumentation is in good shape and ready for ramps.
Dec. 9 DX training is finished. It took 3 quenches to get to Brho=840 Tm.
            LLRF Commissioning is coming along well.
            They worked all last night and are ready to work on acceleration.
            Injection and capture in blue is well tuned up.
            Orbits are in good shape and tunes and decoupling are set properly.
            Today we prepare the Blue MMPS for ramping.
            This evening we begin working on the ramp.
Dec. 8 Last night went very well. RF Capture was setup in Blue.
            So LLRF Commissioning is coming along well.
            Other work done included instrumentation setup.
            The orbit system is working, as well as the base band tune
            measurement system, beam transfer function measurements, and
            longitudinal monitors.
            Tonight we hope to do the first ramp in Blue.
Dec. 7 Blue injection and circulating beam was established last night.
            DX training was started, continues tonight.
Dec. 6 An attempt to inject beam was made, but problems with the AGS L10 cavity
            and with setup of the new RHIC LLRF prevented it.
            DX training today and tonight.
            We attempt to work on injection again in the early morning hours.
Dec. 5 No beam was injected last night.
            Beam sweeps and p.s. checkout took longer than expected.
            Tongight we work on injection and possibly RF capture.
Dec. 4 As of 6 pm, we remain on schedule.
            RHIC has been swept and secured.
            We expect to inject into Blue later tonight.
Dec. 4 Both Blue and Yellow at 4.5 K.
            X4 Storage RF cavity Damper was repaired.
Dec. 3 "mini-maintenance" this morning.
            Last night beam was re-established to the W-Dump.
            Blue is at 4.5 K, starting cooldown of Yellow.
Dec. 2 RHIC Cooldown remains on schedule. All of Blue is below 10 K.
Dec. 2 On Monday power for the Booster was shutdown (emergency work).
            A side affect was the D3 vacuum valve wouldn't open afterwards.
            (valves were closed as a normal precaution).
            It had to be opened manually.
            As of this morning, beam has not been restored
            due to other equipment failures in the Booster.
Dec. 1 Beam setup in AGS and to AtR dump is in good shape.
            Today cooldown to 4.5 K begins in Blue. 1st 9 am meeting.
Nov. 30 Put together plan for next week or so; see Plan from Dec.1 to Dec.8
Nov. 24 Working on merge setup in AGS.
Nov. 22 Beam has been extracted from the AGS to the W-dump.
Nov. 19 Today is an important day, LLRF begins switchover to the new system!
Nov. 19 Looks like the last day of the Dry Run. Most tasks are completed.
Nov. 16 completed first day of Dry Run. Cryo on schedule.