# Shuttling gone wrong

If a revert in Production is required, stemming from the shuttle of any repository, this is your guide!

# Reverting

Do not be afraid to revert your code if anything goes unexpected. It is better for all our users and developers alike.

Start by following the specific revert guides if they exist (Gutenberg, MarfeelXP)

If you have no specific guide, the process boils down to 2 points:

  1. Revert : Don’t fix the issue and deploy, it takes longer and the impact is higher.
    • Revert in Jenkins by shuttling the previous build first, and then do the fix in Github
  2. Communicate : It is KEY that Support teams are aware of incidents before clients.

Finally, delete the shuttling calendar event that you used. This way, the Shuttles calendar only shows a history of what is in production.

# Revert communication

Send an email summarising the issue to pem@marfeel.com using the template below explaining the problem.

WARNING

Do not wait until everything is working to send it. You can always send a follow-up to your first email once more information was uncovered.

**Epic**: Test Gemini on tenants with their own Prebid.

**[ Build | Upgrade | Change | Cause ]:** Technical details are not necessary

* XP build 1234
* GTB build 3424
* MG MContigo build 3324
* Upgrade MongoDB version

**Effect**: (CS should understand it and it should explain how the client will perceive this error)

* adServer not being loaded properly
* Home not refreshing content
* No ads in mosaic
* redirect to the client version

**Impact**:

* Tenant: [www.example.com](http://example.com/)
* All AMP traffic
* All BLG tenants

# Alarms handling

At this point, it is assumed that you understand Alarms System and Anomalies Monitoring (opens new window). If not please redirect to Marfeel Alarms section in Atenea (opens new window).

# Perform Triage

  • Review new alarms next 30-60 mins after your deploy
  • New alarms can be identified in the central dashboard as:
    • Priority: Undefined
    • Team: Undefined

# Move Alarm to ‘In Progress’

As soon as you start investigating the alarm, move it to In Progress and assign it to yourself.

This makes it clear for everyone that someone is working on it.

# Change issue priority if necessary

Especially with critical issues

  • If after investigating the issue is clear for you that there’s no impact, change the priority accordingly

# Close the Alarm ASAP

Especially with critical issues because if not support team is blind.

Once the fix is done, if errors are stable, close the issue (Evaluate is NOT an Option).

TIP

While an alarm is active, it silences the alarming mechanism for that type of error. This is good to avoid receiving a new ticket every few minutes related to the same issue, but it means that any newly introduced error goes undetected.

Always close your alarm as soon as possible to keep the system running smoothly.