xiven.com stating the blatantly obvious since 2002

Archive

View: 2016, Jan, Feb, Mar, Apr, May, Jun, Jul, Aug, Sep, Oct, Nov, Dec, 2018, By category, Full index

Viewing entries for August 2017

SSH Key Authority demo server

Following up on the recent open source release of our SSH Key Authority tool, there is now a demonstration server available. You can find it here:

SSH Key Authority demonstration server

Use one of the following sets of username / password credentials to log in:

Posted: 2017-08-31 22:05:17 UTC by Xiven | Cross-references (0) | Comments (0)

Restoring a deleted custom field in JIRA from backups

Imagine if you will, the hypothetical scenario where your newest JIRA admin has just managed to delete a custom field from your JIRA instance. This of course means that all data stored in that field has been deleted across all issues in all of your projects, and all the schemes and configurations it was referenced by will no longer reference it.

So, after revoking this hypothetical admin's administration privileges, what is your next step?

Like any good system administrator, you obviously have full database backups running on at least a nightly basis, so almost all of the data is there and can (at least theoretically) be recovered. But you don't want to restore the entire database, otherwise you would lose all data for today, which would be very bad.

You could re-create the field through the JIRA UI and then restore the data into this new field, but that has a couple of caveats:

It would be much better if you could restore the field and all of the configuration surrounding it directly as it was before. But how will you know which data to restore, and how will you actually perform this restoration? Allow me to be your guide...

Step-by-step guide

This guide is written for JIRA admins who are using a PostgreSQL database as the backend of their JIRA instance. If you're using a different database engine you will need to make various adjustments.

  1. First things first, get that database backup file and copy it to some server or workstation where you can safely mess around with a database without consequence.
  2. On this safe place, import the file into your database: psql jira < 2017-08-31.sql
  3. Connect to postgres: psql jira
  4. Find the ID of your now-deleted field: select id from customfield where cfname = 'My custom field'; (for this hypothetical scenario, let's assume the ID was 10141).
  5. You'll also need to find the ID of any entries in the fieldconfiguration table that reference your field: select * from fieldconfiguration where fieldid = 'customfield_10141'; (we'll assume we got an ID from this query of 10131)
  6. Copy out the data from the relevant tables, filtered specifically to your custom field:
    
    create table cf_temp as select * from customfield where id = 10141;
    create table cfo_temp as select * from customfieldoption where customfield = 10141;
    create table cfv_temp as select * from customfieldvalue where customfield = 10141;
    create table fsli_temp as select * from fieldscreenlayoutitem where fieldidentifier = 'customfield_10141';
    create table cli_temp as select * from columnlayoutitem where fieldidentifier = 'customfield_10141';
    create table fli_temp as select * from fieldlayoutitem where fieldidentifier = 'customfield_10141';
    create table cc_temp as select * from configurationcontext where customfield = 'customfield_10141';
    create table fc_temp as select * from fieldconfiguration where fieldid = 'customfield_10141';
    create table gc_temp as select * from genericconfiguration where datatype = 'DefaultValue' and datakey = '10131';
    create table fcsit as select * from fieldconfigschemeissuetype where fieldconfigscheme = 10131;
    create table fcsit_temp as select * from fieldconfigschemeissuetype where fieldconfigscheme = 10131;
    
  7. Now you'll want to extract this data from postgres in the form of SQL INSERT statements. From your shell command-line:
    
    pg_dump --data-only --inserts -t cf_temp jira > cf.sql
    pg_dump --data-only --inserts -t cfo_temp jira > cfo.sql
    pg_dump --data-only --inserts -t cfv_temp jira > cfv.sql
    pg_dump --data-only --inserts -t fsli_temp jira > fsli.sql
    pg_dump --data-only --inserts -t cli_temp jira > cli.sql
    pg_dump --data-only --inserts -t fli_temp jira > fli.sql
    pg_dump --data-only --inserts -t cc_temp jira > cc.sql
    pg_dump --data-only --inserts -t gc_temp jira > gc.sql
    pg_dump --data-only --inserts -t fc_temp jira > fc.sql
    pg_dump --data-only --inserts -t fcs_temp jira > fcs.sql
    pg_dump --data-only --inserts -t fcsit_temp jira > fcsit.sql
    
  8. These SQL files are almost what you need, but they'll be using the wrong table names. Use some simple sed magic to clear those right up:
    
    sed s/cf_temp/customfield/ < cf.sql > cf-out.sql
    sed s/cfo_temp/customfieldoption/ < cfo.sql > cfo-out.sql
    sed s/cfv_temp/customfieldvalue/ < cfv.sql > cfv-out.sql
    sed s/fsli_temp/fieldscreenlayoutitem/ < fsli.sql > fsli-out.sql
    sed s/cli_temp/columnlayoutitem/ < cli.sql > cli-out.sql
    sed s/fli_temp/fieldlayoutitem/ < fli.sql > fli-out.sql
    sed s/cc_temp/configurationcontext/ < cc.sql > cc-out.sql
    sed s/gc_temp/genericconfiguration/ < gc.sql > gc-out.sql
    sed s/fc_temp/fieldconfiguration/ < fc.sql > fc-out.sql
    sed s/fcs_temp/fieldconfigscheme/ < fcs.sql > fcs-out.sql
    sed s/fcsit_temp/fieldconfigschemeissuetype/ < fcsit.sql > fcsit-out.sql
    
  9. Copy these SQL files over to your JIRA server.
  10. Back up your JIRA database! Trigger an extra special run of your trusty database backup script.
  11. Stop your JIRA instance.
  12. Run all of these shiny SQL files:
    
    psql jira < cf-out.sql
    psql jira < cfo-out.sql
    psql jira < cfv-out.sql
    psql jira < fsli-out.sql
    psql jira < cli-out.sql
    psql jira < fli-out.sql
    psql jira < cc-out.sql
    psql jira < gc-out.sql
    psql jira < fc-out.sql
    psql jira < fcs-out.sql
    psql jira < fcsit-out.sql
    
  13. Start your JIRA instance up again.
  14. Check to see if everything looks okay.
  15. Hopefully breathe a huge sigh of relief.
  16. Set a re-index going, just for good measure. This is JIRA we're talking about after all.

Posted: 2017-08-31 17:15:36 UTC by Xiven | Cross-references (0) | Comments (0)

The Keys System

Several years ago, when I was still working in the Core department at Opera Software, I became aware that the Information Services team had started using a web-based tool to manage SSH key access to root accounts on their servers (I discovered this when I needed to upload my key there to get access to a server that was shared with them).

A few reorganizations and ownership shifts later, I found myself in the team that owned this tool, known only as "the Keys system", or by its hostname "keys".

In November 2013 I embarked on a project to rewrite the Keys system, adding the ability to manage more than just the root account, and improving the user interface. Other features were added later such as group access management, SSH access options, and immediate syncing. It soon became quite well-liked within the company as a tool for people to manage access to their servers.

We've been hoping to open source this for quite some time, and after the successful open-sourcing of our PowerDNS management tool (which was actually developed after the Keys system and shares a lot of core code in common) we've been getting ready to go ahead with this one.

One of the big challenges was to come up with an actual name for the project. In the end we settled on "SSH Key Authority". Today we have released SSH Key Authority to the public on GitHub!

Posted: 2017-08-22 16:10:16 UTC by Xiven | Cross-references (1) | Comments (0)