A Github repository issue backup tool developed by ABAP

I personally prefer to use Github repository issue to manage my knowledge. For issue, this is my github repository to store ABAP and SAP CRM related knowledge of mine and currently it has already 285 issues.

There are two main reasons which drive me to choose github repository as my knowledge management approach:

• great speed of full text search
• powerful and flexible label management of issues

As the accumulation of issue number, I have to consider the possibility of issue backup, so that I can still have access to my knowledge repository when the public github is not available ( for example due to network issue or access within China banned by government due to political reasons, who knows ).

However after I go through them, I found none of the tool can 100% fulfill my backup requirement, as a result I plan to write one backup too on my own using github public API documented here.

Basic idea of this tiny issue backup tool implemented by ABAP

1. Issues should be persisted to ABAP server

Since as an ABAPer we have powerful Netweaver behind us, I plan to develop a tool which can extract all issues of a given repository and stored them in an ABAP system, so that I can use all kinds of ABAP tools for example CDS view to make analysis on those issues later.

Only a single API endpoint is used in this tool, see how to use it below.

The format of this endpoint url is:

https://api.github.com/repos/<your github user id>/<name of github repository>/issues

In my example above, the response is a JSON string which is an array with each element representing an issue detail.

For me I am only interested

with issue number, issue title, issue body text, created and update timestamp, so I create the following database table to store github issue information:

1.2 Delta-backup functionality support

In github repository each issue is marked with a number, which could be easily found in issue url and issue detail page.

The delta backup function means for example I have finished the backup of current 285 issues, and later when I create two new issues 286 and 287, and then I run the tool again, only 286 and 287 should now be exported and persisted into ABAP system.

1.3 Paging in issue API needs to be considered

In current Github REST API V3, by default only 30 issues are returned in a single API call.

If there are still left issues to be exported, there will be two indicators stored in HTTP response fields of this single API call, one (rel=”next”) contains url which returns the next 30 issues and the other(rel=”last”) contains the url which returns the issues in the last page ). This paging logic should also be supported in the tool.

1.4 API Rate Limiting is not considered for simplification reason

How to use this tool

1. Create one transparent table to store issue detail

And fill one entry for the github repository whose issue you would like to extract accordingly.

You can choose any name in column REPO_NAME which acts as an internal key for the github repository name.

2. execute method START_BACKUP of tool class, specify the internal repository name you specified in the first step.

Once finished, you could find that all issues are stored into table CRMD_GIT_ISSUE.

3. The body of a given issue could be downloaded locally using method DOWNLOAD_ISSUE, just specify the repository name, issue number and the path of local folder where the github issue file you would like to store:

Since most of my github issue contains Chinese characters, so I use code page 8400 in the code.

here below is the original issue opened in the browser:

Here below is the downloaded issue source code in markdown format:

More features to be added

I will enhance the tool with following features in the future

3.1 issue label information

Multiple labels could be maintained for a single issue and it is very convenient to search by these maintained labels.

Currently the label information is not stored in the database table.

3.2 CDS views for analysis purpose

Since now we have lots of content in our database table, we could develop several CDS view to meet with our specific requirement, for example:

• calculate the total length of all issues belonging to a given github repository
• get an overview of all labels and the concrete number of issues assigned with each label

3.3 image backup

Currently only the issue source code with markdown format is extracted and stored in database table in ABAP server. However most of my issues contains screenshot and in the issue source code only a reference to these image files are there – the binary content of image files themselves are not stored in ABAP server. This could be enhanced in the future.

The source code of the tool could be found from my github.