My GSOC adventure with XWiki and Dokuwiki

Version 4.1 by Neha Gupta on 2018/08/13

Aug 13 2018

Warning
This blog post is not published yet.

Mentors :

Thomas Mortagne, Shubham Jain

A little bit about me

I’m a final year B.Tech student, from Bharati Vidyapeeth's College of Engineering, India. I’m 22 years old and I feel pleased to be a part of XWiki community as a GSoC’18 student and learn more about both XWiki and Dokuwiki while playing with Java. 

Description

The project focused on improving the existing DokuWiki importer that imports instances of DokuWiki to XWiki by using some intermediate common events based on Filter Stream Framework. In the previous Dokuwiki importer module already supported basic functionalities. Improvements like support for handling unserializing of files with no metadata, lists, image-link, interwiki-link, macro support and other syntax-parser bug resolution.

You can get inspiration from

http://extensions.xwiki.org/xwiki/bin/view/Extension/Import+DokuWiki+into+XWiki+Application and

Documentation can be found at

 https://extensions.xwiki.org/xwiki/bin/view/Extension/DokuWiki

The issue tracker can be found

 http://jira.xwiki.org/projects/DOKUWIKI/issues/DOKUWIKI-11?filter=allopenissues

Developer Profile

Java

Status

Successfully terminated

Milestones

Milestone 1 [May 14 - June 15]:

Blocking bug fixing and folder support.

I fixed the warning and critical bugs, by making modifications in dokuwiki-text module. This also included setting up the project and getting used to the Component architecture of XWiki.

  • Refactoring my understanding of the Filter API. 
  • Getting used to the Component architecture of XWiki. 
  • Finding the ideal way to check for input-source type and add support for  DokuWiki data-directory as an input source.
  • Understanding Issues with apache compress behavior difference with file input and streaming input. Finding the bug in FileUtils.CopyToFile (closing streams after copying the file) and resolve the FileNotFound Blocking Bug.
  • Fixing the crash of Dokuwiki importer after a few Seconds of start with class ArrayIndexOutOfBoundsException.
  • The existing code ended up in an infinite loop when importing a tgz input-source package, so the exception handling was added for it to try to find the type of the file before actually read it, before it tried to read it as a zip directly and expect the first read to fail if it's not one basically.
  • Tests were written, and post-testing, the extension was released on XWiki nexus repository. 
  • Finally, documentation was modified on the XWiki extension page. 

Milestone 2 [May 14 - June 15]:

 

Adding more syntax features.

The next step was to modify the DokuWiki-syntax parser module and add support for some more popular features and fixing bugs in image-syntax parsing. The idea was to refine the most of the basic syntax available in DokuWiki. 

  • Thorough understanding of the DokuWiki syntax. Critical differences between XWiki and DokuWiki syntax and macros. 
  • Understand the XWiki rendering API.
  • Understanding occasional differences in behavior in the XML input and the output string from xwiki/2.1 syntax renderer. 
  • Added support for DokuWiki Interwiki links.
  • Added support for DokuWiki image links.
  • Added support for DokuWiki indented lists.
  • Fixed the bug of no alignment of images. In the previous module, the image feature support checked for image alignment to be center, left or right but, doesn't return the alignment parameter in the output, so all the images were aligned right. 
  • Test cases were added for the new features.
  • The changes in parser were released as an extension on XWiki nexus repository. 

Milestone 3 [July 20 - August 14]:

Adding macros framework.

The next step was to parse the DokuWiki syntax-plugins and call the relevant methods of the Rendering API so that the output is the XWiki syntax. The idea was to add a DokuWiki macro-framework.

  • AngleBracketDokuWikiPlugin and CurlyBracketDokuWikiPlugin components were implemented which has DokuWiki plugin role which parses the source input and checks for the type of plugin if no syntax-plugin entry point match it returns the stateful buffer.
  • Components in angleBrackets and curlyBrackets package implements their functional interfaces like DokuWikiAngleBracketPlugin and DokuWikiCurlyBracketPlugin which parse the input source if macro/plugin pattern is detected, by checking for each plugin component in the same pattern.
  • DokuWikiSyntaxParserHelper is the helper class which implements common helper function for both the plugin framework and the syntax-parser module.
    Finally, an updated version of the extension was released on the XWiki nexus repository. 

Deliverables

  • Add improvements to support for ordered, unordered and nested lists. [ Issue 6 ]
  • Add code for image-links supporting links to other internal or external pages. [ Issue 5 ]
  • Add support for a folder as a source input. [Issue 8]
  • Handling un-serialising of files with no metadata. [ Issue 2 ]
  • Support for Interwiki-link to move to other wikis. [ Issue 4 ] 
  • Fix OOM when importing a tgz. [Issue 1]
  • Working on FileNotFoundException error. [ Issue 3 ]
  • Add popular macros support with and without arguments. [ Issue 7 ]
  • Optimise the way to support various archive type.
  • No alignment of image detected. [ Issue 9 ]

First Milestone release, latest version here

Second Milestone release, latest version here

Third Milestone release, latest version here

The extension can be downloaded from here

 

Get Connected