twindexer

This is a basic tool I wrote initially fo automating the indexing of posts in my gemini capsule at rawtext.club. This was the first thing I ever wrote in Python, and even though it was very crude and hacky, several brave souls actually made use of it to manage their own capsules. So I have since refactored the code and made it a little more versatile.

What can it do?

Twindexer was built to generate gemlog indexes in which the posts are categorized into different "tags," or topics. The "tags" are just folders where the posts of that tag live. Twindexer finds all the world-readable posts in these folders, writes individual indexes for each of them, and then writes a main gemlog index containing all of them--nicely formatted with a tag label in the link-line of each post. Twindexer is now more configurable, allowing some more flexibility about how your capsule is structured and how your indexes are formatted. It is a very basic utility, but some amenities include:

• Specify headers and footers for indexes as either a path to a file or a multi-line string within the config file itself.
• Titles, labels, and folder names of tags can all be different
• Index files can be named an located anywhere in the capsule
• Manage multiple capsules using any number of separate config files, which can be located anywhere, even inside your capsule!
• Optionally generate an atom feed for each index.

Installation and Use

Just clone this repo, create a configuration file (see below), and pass its path to the script using the -c flag. Twindexer uses PyYAML to parse its configuration files, so be sure you have it installed in your environment (pip install PyYAML).

WARNING!

Twindexer will overwrite existing data without asking! So if you want to use twindexer to manage an existing capsule, it is wise copy its contents to a test folder and create a test config file that points to it.

Capsule Structure

Twindexer assumes that posts for each tag live in a folder specific to that tag. The location and name of the gemlog and tag indexes can be anything/anywhere within the capsule. Filenames of posts must begin with a date in the form of YYYY-MM-DD. This date will be used as the published date in the indexes. The script also assumes that you title your posts with a top-level heading! Otherwise your indexes will be...barely indexes...

Configuration

The configuration file is in YAML format, and can be nammed and located anywhere on your system. The structure is as follows:

capsule:
  baseURL:
  gemroot:
  author:
  email:
  license:
gemlog:
  title:
  index:
  feed:
  header: 
taglist: 
  tag_name:
    title: 
    index:
	folder:
    tag:
    separator:
    feed:
    header:
    footer:
  tag_name:
    title: 
    index:
	folder:
    tag:
    separator:
    feed:
    header:
    footer:
  tag_name:
    ...
    ...

Global Variables

• baseURL: the base url of capsule (eg. gemini://example.com/~user/), used for feed generation
• gemroot: root directory of capsule (on machine where twindexer runs).
• author: your name/pseudonym (used for feed generation)
• email: your email address (used for feed generation, can be formatted however you like, e.g. mieum(at)rawtext.club)
• license: e.g. CC-BY-SA 4.0 (used for feed generation

Gemlog Variables

• gemlog: section containing key-value pairs pertaining to gemlog index
• title: optional but recommended title of gemlog
• index: path to the gemlog index file, relative to gemroot.
• feed: boolean value; optional. If True, twindexer will create a link to an atom.xml file in the same folder.
• header: optional; can be a path to a file or a multiline string, or, block literal scalar in YAML jargon.
• footer: same as header

Tag Variables

• taglist: everything nested under the taglist is a tag/folder in your capsule
• tag_name: name of your tag. this key can be called anything, twindexer does not reference it directly, it is mostly there for convenience.
• title: title of tag, used in indexes, feeds, etc.
• index: path to the index for the tag, relative to gemroot, trailing slash (e.g. tag/)
• folder: folder containg all the posts for the given tag, relative to gemroot, trailing slash
• tag: the string to be used as a label for the tag (used in link line)
• separator: string used to separate date and title in linklines on tag indexes. defaults to "-" if not given.
• feed, header, footer: same as gemlog

As with all things YAML, mind the whitespace/indentation!

TIP: values can be reused using the anchor(&) and alias(*) notation. See sample config for an example.

Sample Configuration

This is the configuration live on my capsule at the time of writing this. An updated version can be found on my capsule itself at rawtext.club/~mieum/twindexer.yml

capsule:
  baseURL: gemini://rawtext.club/~mieum/
  gemroot: ~/var/twindexer/
  author: mieum
  email: mieum(at)rawtext.club
  license: 'CC-BY-NC-SA 4.0'
gemlog:
  title: mieum's rawtext gemlog
  index: gemlog.gmi
  feed: False
  header: |
    ```
    ███████████████████▎██████▎▅████████████████▅
    ███████████████████▎██████▎███▎▎    █████████ ▎
    █████▎━━━━━━███████▎██████▎▀████████████████▀ ▎
    █████▎▎     ███████▎██████▎▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅
    █████▎▎mieum███████▎██████▎██████████████████ ▎
    ███████████████████▎██████▎▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅▅
    ███████████████████▎██████▎██████████████████ ▎
    ███████████████████▎██████▎█████████▎▎     ██ ▎
    ███████████████████▎██████▎██████████████████ ▎
     ━━━━━━━━━━━━━━━━━━━ ━━━━━━ ━━━━━━━━━━━━━━━━━━
    ```
       
  footer: |
    
    _________
    => ./index.gmi home
    CC-BY-NC-SA 4.0 mieum(at)rawtext.club
taglist:
  poems: 
    title: verse
    index: poems/index.gmi
    folder: poems/
    tag: 詩
    feed: True
    header: |      
      ```
          , /\  ' . (    `      *
       *   //\\`  ___ '   /\  .    '
          ///\\\ /   \_ */  \_/\  ,
      .  ////\\\\  /\  \/       \ _  `
        //////\\\\//\\    _  /\  //\
       ////////\\\\/\\\ /\\\///\////\
      /////////poems\\\/\\\\\///\////\
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      ||||||||||||||||||||||||||||||||
        ||||||||||||||| |||| ||||||||
       `  |||||    || .       |  ,||
      ```
         
    footer: &footer |
      
      _________
      => ../gemlog.gmi gemlog
      => ../index.gmi home
      CC-BY-NC-SA 4.0 mieum(at)rawtext.club
  prose:
    title: ramble, rant, pedant
    index: prose/index.gmi
    folder: prose/
    tag: 散
    feed: True
    header: |
      ```
            /\                     )
           //\\  /\        /\
          ///\\\/ \\  /\  //\\   /\    /\
         ////\\\\  \\//\\///\\\ // \  //\\
         ////\\\\ \ ///\\\//\\\//  \\///\\\
        /////\\\\\ ////\\\\/\\\\  /\ ///\\\
       //////\\\\\\////\\\\/\\\\\//\\///\\\\
       //////\\\\\\////\\\\\|\\\\//\\///\\\\
      ///////\\\\\\\ |||| ,|||  ///\\\ || .
        '  ||||   ,   .       .   || ' .
      .  '     .         `       `   ,    '
      ```
         
    footer: *footer
  relog:
    title: RElog
    index: relog/index.gmi
    folder: relog/
    tag: 答
    feed: True
    header: |
      ```
       ____ ____ ____ ____ ____
      ||R |||E |||L |||O |||G ||
      ||__|||__|||__|||__|||__||
      |/__\|/__\|/__\|/__\|/__\|
      ```
         
    footer: *footer
  music:
    title: mieum's musical mythoi
    index: music/index.gmi
    folder: music/
    tag: 樂
    feed: True
    footer: *footer

Roadmap

Some things I am working on implementing:

• gempub archive generation for each index and entire capsule
• management of non-tag pages (feed generation, indexing, etc.)