Viewing entries for August 2017
SSH Key Authority demo server
Following up on the recent open source release of our SSH Key Authority tool, there is now a demonstration server available. You can find it here:
Use one of the following sets of username / password credentials to log in:
- testuser / testuser - normal user with admin access granted to a few servers
- testadmin / testadmin - admin user
Restoring a deleted custom field in JIRA from backups
Imagine if you will, the hypothetical scenario where your newest JIRA admin has just managed to delete a custom field from your JIRA instance. This of course means that all data stored in that field has been deleted across all issues in all of your projects, and all the schemes and configurations it was referenced by will no longer reference it.
So, after revoking this hypothetical admin's administration privileges, what is your next step?
Like any good system administrator, you obviously have full database backups running on at least a nightly basis, so almost all of the data is there and can (at least theoretically) be recovered. But you don't want to restore the entire database, otherwise you would lose all data for today, which would be very bad.
You could re-create the field through the JIRA UI and then restore the data into this new field, but that has a couple of caveats:
- You have to manually re-integrate the field into all the field configurations and screens that it used to be in.
- The ID of the field will have changed, so if the field is referenced by any workflows, scripts or filters by ID then all of these things will need to be fixed manually.
It would be much better if you could restore the field and all of the configuration surrounding it directly as it was before. But how will you know which data to restore, and how will you actually perform this restoration? Allow me to be your guide...
This guide is written for JIRA admins who are using a PostgreSQL database as the backend of their JIRA instance. If you're using a different database engine you will need to make various adjustments.
- First things first, get that database backup file and copy it to some server or workstation where you can safely mess around with a database without consequence.
On this safe place, import the file into your database:
psql jira < 2017-08-31.sql
Connect to postgres:
Find the ID of your now-deleted field:
select id from customfield where cfname = 'My custom field';(for this hypothetical scenario, let's assume the ID was 10141).
You'll also need to find the ID of any entries in the
fieldconfigurationtable that reference your field:
select * from fieldconfiguration where fieldid = 'customfield_10141';(we'll assume we got an ID from this query of 10131)
Copy out the data from the relevant tables, filtered specifically to your custom field:
create table cf_temp as select * from customfield where id = 10141; create table cfo_temp as select * from customfieldoption where customfield = 10141; create table cfv_temp as select * from customfieldvalue where customfield = 10141; create table fsli_temp as select * from fieldscreenlayoutitem where fieldidentifier = 'customfield_10141'; create table cli_temp as select * from columnlayoutitem where fieldidentifier = 'customfield_10141'; create table fli_temp as select * from fieldlayoutitem where fieldidentifier = 'customfield_10141'; create table cc_temp as select * from configurationcontext where customfield = 'customfield_10141'; create table fc_temp as select * from fieldconfiguration where fieldid = 'customfield_10141'; create table gc_temp as select * from genericconfiguration where datatype = 'DefaultValue' and datakey = '10131'; create table fcsit as select * from fieldconfigschemeissuetype where fieldconfigscheme = 10131; create table fcsit_temp as select * from fieldconfigschemeissuetype where fieldconfigscheme = 10131;
Now you'll want to extract this data from postgres in the form of SQL
INSERTstatements. From your shell command-line:
pg_dump --data-only --inserts -t cf_temp jira > cf.sql pg_dump --data-only --inserts -t cfo_temp jira > cfo.sql pg_dump --data-only --inserts -t cfv_temp jira > cfv.sql pg_dump --data-only --inserts -t fsli_temp jira > fsli.sql pg_dump --data-only --inserts -t cli_temp jira > cli.sql pg_dump --data-only --inserts -t fli_temp jira > fli.sql pg_dump --data-only --inserts -t cc_temp jira > cc.sql pg_dump --data-only --inserts -t gc_temp jira > gc.sql pg_dump --data-only --inserts -t fc_temp jira > fc.sql pg_dump --data-only --inserts -t fcs_temp jira > fcs.sql pg_dump --data-only --inserts -t fcsit_temp jira > fcsit.sql
These SQL files are almost what you need, but they'll be using the wrong table names. Use some simple
sedmagic to clear those right up:
sed s/cf_temp/customfield/ < cf.sql > cf-out.sql sed s/cfo_temp/customfieldoption/ < cfo.sql > cfo-out.sql sed s/cfv_temp/customfieldvalue/ < cfv.sql > cfv-out.sql sed s/fsli_temp/fieldscreenlayoutitem/ < fsli.sql > fsli-out.sql sed s/cli_temp/columnlayoutitem/ < cli.sql > cli-out.sql sed s/fli_temp/fieldlayoutitem/ < fli.sql > fli-out.sql sed s/cc_temp/configurationcontext/ < cc.sql > cc-out.sql sed s/gc_temp/genericconfiguration/ < gc.sql > gc-out.sql sed s/fc_temp/fieldconfiguration/ < fc.sql > fc-out.sql sed s/fcs_temp/fieldconfigscheme/ < fcs.sql > fcs-out.sql sed s/fcsit_temp/fieldconfigschemeissuetype/ < fcsit.sql > fcsit-out.sql
- Copy these SQL files over to your JIRA server.
- Back up your JIRA database! Trigger an extra special run of your trusty database backup script.
- Stop your JIRA instance.
Run all of these shiny SQL files:
psql jira < cf-out.sql psql jira < cfo-out.sql psql jira < cfv-out.sql psql jira < fsli-out.sql psql jira < cli-out.sql psql jira < fli-out.sql psql jira < cc-out.sql psql jira < gc-out.sql psql jira < fc-out.sql psql jira < fcs-out.sql psql jira < fcsit-out.sql
- Start your JIRA instance up again.
- Check to see if everything looks okay.
- Hopefully breathe a huge sigh of relief.
- Set a re-index going, just for good measure. This is JIRA we're talking about after all.
The Keys System
Several years ago, when I was still working in the Core department at Opera Software, I became aware that the Information Services team had started using a web-based tool to manage SSH key access to root accounts on their servers (I discovered this when I needed to upload my key there to get access to a server that was shared with them).
A few reorganizations and ownership shifts later, I found myself in the team that owned this tool, known only as "the Keys system", or by its hostname "keys".
In November 2013 I embarked on a project to rewrite the Keys system, adding the ability to manage more than just the root account, and improving the user interface. Other features were added later such as group access management, SSH access options, and immediate syncing. It soon became quite well-liked within the company as a tool for people to manage access to their servers.
We've been hoping to open source this for quite some time, and after the successful open-sourcing of our PowerDNS management tool (which was actually developed after the Keys system and shares a lot of core code in common) we've been getting ready to go ahead with this one.