Mueez Khan
Mueez Khan
Mueez Khan
A version of this blog post has also been republished on datHere's blog.
Let's walk through the process of a project I worked on: setting up an interactive timeline based on the CKAN data management system's version release history. You may view the timeline here and an embedded recap of its presentation below.
CHANGELOG.rst file in the GitHub repository for CKAN consolidating all release versions including each of their dates and changes made in a more readable format. The file is written in a format similar to Markdown, so I decided to build a Python script.
CHANGELOG.rst data into CSV format which we could import into our spreadsheet.
import csv
import markdown
# Generate a list from a file for importing into a CSV
def extract_data(file_path):
# Open the .rst file
with open(file_path, 'r') as file:
# Generate a string list of each line from the .rst file
lines = file.readlines()
# List of each release's data to import into CSV
data = []
# Row values for each column in the CSV
year = ''
month = ''
day = ''
headline = ''
text = ''
# Loop through each line in the .rst file
for line in lines:
# Remove starting and ending whitespaces (if any)
line = line.strip()
# Line starts with v or v. followed by a number (v2 or v.2)
if (line.startswith('v') and line[1].isdigit()) or (line.startswith('v.') and line[2].isdigit()):
# Append the accumulated data for the previous version to the list
data.append([year, month, day, headline, text])
# Reset the text for writing the current version's changes
text = ''
# Extract the release version and date information
headline = line.split()[0]
datetime = line.split()[1]
year, month, day = datetime.split('-')
# Ignore lines starting with '=' (section headers)
elif line.startswith('='):
continue
# Accumulate the lines between release versions into the text variable
else:
# Convert the current line to HTML
text += markdown.markdown(line) + '\n'
# Append the last release version's data to the data list
data.append([year, month, day, headline, text])
return data
# Write the extracted release data to a CSV file
def write_to_csv(data, output_file):
# Create or overwrite a CSV file
with open(output_file, 'w', newline='') as file:
# Instantiate a CSV file writer object
writer = csv.writer(file)
# Write the header row in the same format as the Google Sheet
writer.writerow(['Year', 'Month', 'Day', 'Headline', 'Text'])
# Write each release to the CSV file as a row
for row in data:
writer.writerow(row)
# Example usage
file_path = 'CHANGELOG.rst'
output_file = 'output.csv'
# Extract data from the .rst file
file_data = extract_data(file_path)
# Write the data to a .csv file
write_to_csv(file_data, output_file)Text field, which includes the description for a release of CKAN. Therefore, I used the markdown package to convert each line into HTML format since the input was akin to Markdown styling.
.rst format or for rendering the data on a changelog page on CKAN's docs site) that were not covered in the conversion properly.<ul> for each <li>), though this is fine since we can reuse the script by writing the release description in Markdown if revising it.day value so they were removed from CHANGELOG.rst and manually inputted into the timeline (though this can be fixed by editing the script).CHANGELOG.rst file, so any modifications made would need to be updated in the Google Sheet including new version releases. An automation system is definitely possible though.Updates made to this blog post over time.
Submit a comment about this post that may be shared.
Receive updates during my career.
Technical writing on work experiences, projects, and tools you may find useful.