You are not logged in.

#1 2012-02-28 02:54:15

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Xac: a markdown and Pandoc-based content management system

Woohoo, haven't had time to post a contribution thread in ages...

I finally got around to fixing the upload scripts for my site and ended up completely rewriting the content management system. The old one was a Frankenstein's monster of bash, perl, pandoc, php and about 5 metric tons of sh..., um, kludges.

The new Xac is much simpler. It's all written in (relatively clean) Python 3. It  takes files from different source directories, processes markdown through pandoc, interpolates some files if you want, and publishes everything to another directory that you can serve. The look and feel is managed through Pandoc output templates and can be XHTML, PHP, etc. It supports arbitrary plugins via simple Python functions (which can be provided by modules for more complex functionality).

The site itself is the example, and a working default is provided in the source files. If you want to check it out, start by downloading the source file and reading the README to get the server up (it basically just tells you to run serve.sh). You can view the default server's index page online too. The default server is admittedly a bit complex, but that's because it is what I actually use for the site (although I have some custom functions for managing my repos and other things). You can layer any level of complexity on top of the base functionality because it's all in Python (i.e. not an arbitrary configuration language), but you can keep it dead-simple too. The source archive includes a minimal example too.

I'll continue to improve code and documentation as time goes by. If you have any questions, just ask.


Before anyone asks why they should use Xac, read this.



Incidentally, the site feels much snappier now with all that nasty PHP gone.

Offline

#2 2012-07-16 14:55:14

manolomartinez
Member
Registered: 2011-04-29
Posts: 64
Website

Re: Xac: a markdown and Pandoc-based content management system

Hi, I'm toying with the idea of substituting my WP blog/site with xac. I was wondering if you have an example of a minimal blog setup. Basically, posts with the possibility of comments. If you don't have a worked out example, but can provide pointers that might help me develop it, please let me know too.

Cheers,
Manolo

Offline

#3 2012-07-16 23:09:49

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Xac: a markdown and Pandoc-based content management system

The default server in the source archive includes a simple news feed example. I use the same functionality for blog posts on my site. It may be a bit too simple for what you want, but if you know Python then you can use the news feed plugin module to create your own fully functional blog module. If you have specific questions I will try to answer them.

Btw, the current news/blog plugin auto-archives articles after a given interval (you can see it on my site). I intend to update it with an option to always keep at least x items in the main directory.

Comments could be implemented by using a custom template. For example,  you could create a Pandoc template named "blog.php" (in "files/templates/")  with support for a database-backed PHP comments section. Check the Pandoc documentation for information about Pandoc templates (example: files/templates/main.html). You can control which template file is used for a given path with the "get_pandoc_template" parameter (example: publish_simple.py).

You will probably need to insert a database ID into each template for the code to work. You can do that with Pandoc variables and "get_pandoc_args".  The basic idea is simple. You can pass in additional variables with "-V foo=bar" to have "$foo$" replaced with "bar" in the template file during publishing. The simplest example is in "publish_simple.py", which returns no additional arguments and no input. A somewhat complicated example can be found in "Xac/defaults.py", which supports changing the template banner, the upper and lower navigation links, the footer, etc.



edit:
You could also embed an object  to manage comments in an HTML template, instead of using a full PHP template. I don't know what comment systems are available though, but I expect that there is something generic. I've even considered hosting a FluxBB forum on the site and embedding threads on pages, but that might feel a bit too kludgy.

Last edited by Xyne (2012-07-16 23:39:41)

Offline

#4 2012-07-17 14:02:12

manolomartinez
Member
Registered: 2011-04-29
Posts: 64
Website

Re: Xac: a markdown and Pandoc-based content management system

Thanks a lot, that was useful. I think I'll just start trying to migrate and bug you here if I have questions. Let's see what happens!

Manolo

Offline

#5 2012-07-17 14:34:37

Barrucadu
Member
From: York, England
Registered: 2008-03-30
Posts: 1,158
Website

Re: Xac: a markdown and Pandoc-based content management system

I believe Disqus is used by quite a few sites to implement a commenting system, it might be worth investigating the feasibility of that rather than rolling your own.

Offline

#6 2012-07-17 18:09:41

manolomartinez
Member
Registered: 2011-04-29
Posts: 64
Website

Re: Xac: a markdown and Pandoc-based content management system

Barrucadu, thanks for the suggestion. I've been looking into it for a while and, indeed, implementing a commenting system is far from trivial.

The thing with Disqus is that precisely the whole idea of moving away from WP is to avoid bloat, and to avoid providing third parties with information about my behaviour or the behaviour of visitors to my site (which, otoh, is a very humble blog; hardly slashdot.org).

Offline

#7 2012-07-17 19:15:49

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Xac: a markdown and Pandoc-based content management system

manolomartinez wrote:

implementing a commenting system is far from trivial

Indeed. That's why I still haven't done it. Storing and retrieving comments is easy enough, but authentication, email verification, spam filtering, etc all require some thought.

I expect that there is some easy stand-alone comment system available somewhere than can be plugged in without much hassle, but I have never really looked for one.

I'm definitely interested in this and am looking forward to what you come up with. I may even end up trying some things myself.

Offline

#8 2012-07-23 04:02:33

manolomartinez
Member
Registered: 2011-04-29
Posts: 64
Website

Re: Xac: a markdown and Pandoc-based content management system

Btw,in the latest version of Xac, after running publish.py the side bar appears at the bottom of the page. I'm trying to understand how it all works, and it'd help me if you could take a quick look and post where the bug is (if it's a bug)

Thanks
Manolo

Offline

#9 2012-07-23 14:46:35

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Xac: a markdown and Pandoc-based content management system

When I run publish.py in the current release, I get this: screenshot
I re-released the source archive and double-checked it, so I am quite sure that nothing in Xac has changed.

The layout is controlled by the template and CSS files. Have you changed those? The default templates use a somewhat intricate hierarchy of nested div elements to achieve the layout, so even a very small change in the HTML or CSS can have a prominent affect.

Please post a screenshot if you can so I can see what you see.  If you want some help editing the template files then pastebin what you have changed (or email it to me) and explain what you are trying to do.

If there is something else that is unclear about Xac, just ask. The questions will probably show me where the documentation needs to be improved.

Offline

#10 2012-07-24 02:19:22

manolomartinez
Member
Registered: 2011-04-29
Posts: 64
Website

Re: Xac: a markdown and Pandoc-based content management system

Please find here three different screenshots showing how the test site looks like here after running publish.py twice -- with no changes in what ships in the tar file. The browser is dwb, but I've also tested it on firefox.

So far, I'm just trying to understand the test site. Once I have a clearer idea of what I'm doing, I'll start editing and, for sure, I'll take advantage of your generosity and bug you with my doubts smile

Offline

#11 2012-07-24 08:39:05

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Xac: a markdown and Pandoc-based content management system

Even with dbw I can't reproduce it: dwb screenshot
I tested with several other webkit browsers on a different computer too.

How does the default site look before you run the publish script?
What about my site?

If those look fine then maybe you're missing one of the optional dependencies. The default server showcases several plugins. To use a plugin, you insert a CDATA-like block with named tags into the markdown. The plugin parses that block and replaces it with something else (e.g. a graph). If the plugin isn't supported then that data may get passed through as is. In some cases it might interfere with the HTML.

Can you pastebin or email me the generated pub/index.html page?



Btw, while I was debugging, I noticed that the instructions on the news page were outdated. I have updated it and removed unnecessary scripts from the archive. No extra scripts are necessary to manage the news section.

I have also added support for keeping a given number of entries in the main news directory.

Offline

#12 2012-07-24 14:09:29

manolomartinez
Member
Registered: 2011-04-29
Posts: 64
Website

Re: Xac: a markdown and Pandoc-based content management system

You are right, I didn't have source-highlight installed. I had assumed, wrongly, that that could not be messing with the position of the side bar. Now it looks as it should.

Thanks for the support for keeping a number of entries in the news.

Offline

#13 2012-07-24 15:06:22

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Xac: a markdown and Pandoc-based content management system

manolomartinez wrote:

I had assumed, wrongly, that that could not be messing with the position of the side bar.

It was a reasonable assumption. I have updated the code to escape raw data blocks when a plugin fails. I actually had escaped the outer tags but not the inner data itself.

It's good to have someone else trying things with the code. I only use it in a very limited number of ways so I miss many cases. Thanks for the feedback.

Offline

#14 2012-07-29 04:41:10

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Xac: a markdown and Pandoc-based content management system

New plugin: Aha

The plugin can convert ANSI output to HTML and is therefore useful for displaying (e.g. colorized) command-line output.
You can find an example of armh output here.

The plugin relies on aha, which is available in the AUR: aha-git, aha

Offline

#15 2012-08-09 04:56:26

manolomartinez
Member
Registered: 2011-04-29
Posts: 64
Website

Re: Xac: a markdown and Pandoc-based content management system

I've been looking into this again, and I have some questions. The first one: if I want to have a template specifically for news, I should change defaults.py to something like

  def get_pandoc_template(xac, path):
    """Return path to template file for given path."""
    if path == 'news':
      return os.path.join(TEMPLATES_DIR, 'news.html')
    else:
      if path:
        return os.path.join(TEMPLATES_DIR, 'main.html')
      else:
        return os.path.join(TEMPLATES_DIR, 'listing.html')
  return get_pandoc_template

Right? The code I have posted doesn't seem to work. Am I missing something?

Last edited by manolomartinez (2012-08-09 04:57:05)

Offline

#16 2012-08-24 00:11:35

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Xac: a markdown and Pandoc-based content management system

@manolomartinez
Sorry for the late reply.

defaults.py should not be changed. It is part of the package and should be viewed as a static module. It is included both as an example of more complicated things that you can do with Xac, and as a convenience for me as it allows me to generate my site on different systems.

The path received by the function is the full server path, so "== 'news'" will never match. I would use the following function:

def get_pandoc_template(xac, path):
  """Return path to template file for given path."""
  if path:
    if path.startswith('/news'):
      return os.path.join(TEMPLATES_DIR, 'news.html')
    else:
      return os.path.join(TEMPLATES_DIR, 'main.html')
  else:
    return os.path.join(TEMPLATES_DIR, 'listing.html')

Notice that I have also omitted "return get_pandoc_template" at the end. "get_default_get_pandoc_template" in defaults.py returns a function that returns a path. Here you are directly defining the function that returns the path, and you should define it in your own publish.py file (or in a separate file that it imports).

Offline

#17 2013-01-04 16:48:12

rebootl
Member
Registered: 2012-01-10
Posts: 427
Website

Re: Xac: a markdown and Pandoc-based content management system

Hi

I'm looking for a simple CMS for my personal website and I really appreciate some of the features Xac offers, i.e. the directory listing.

Unfortunately running publish_simple.py returns me an error I don't understand:

[cem@amd64box xac-2012.12.26.1]$ ./publish_simple.py
Traceback (most recent call last):
  File "./publish_simple.py", line 39, in <module>
    get_pandoc_args      = get_pandoc_args,
  File "/home/cem/Setups_amd64box/xac-from-Xyne/xac-2012.12.26.1/Xac/publish.py", line 167, in __init__
    self.watchlist_mtime = max(os.path.getmtime(x) for x in watchlist_files)
  File "/home/cem/Setups_amd64box/xac-from-Xyne/xac-2012.12.26.1/Xac/publish.py", line 167, in <genexpr>
    self.watchlist_mtime = max(os.path.getmtime(x) for x in watchlist_files)
  File "/usr/lib/python3.3/genericpath.py", line 54, in getmtime
    return os.stat(filename).st_mtime
FileNotFoundError: [Errno 2] No such file or directory: '/home/cem/Setups_amd64box/xac-from-Xyne/xac-2012.12.26.1/<frozen>'

I don't think I'm missing any dependencies cause publish.py does work! (Exept for the tex2png plugin that returns me the same error mentioned in the AUR. I tried to rebuild my formats but couldn't get it to work till now. However that's not so important to me cause I don't need it at the moment, but it's also a nice feature.;)

I managed to publish a simple test page using the default publish.py. I'll try to continue with this one and see where I can get.
@Xyne: Thanks for sharing this anyways.

Regards


Personal website: cem.revamp-it.ch
GitHub: github.com/rebootl

Offline

#18 2013-01-04 21:42:58

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Xac: a markdown and Pandoc-based content management system

@rebootl
I was able to reproduce it but I'm still surprised. It seems that some Python module is reporting its path as '<frozen>'. I didn't bother investigating it further and simply added a try/except block to handle such errors. Please try xac-2013.1.

@everyone
I have added a new plugin for inserting file content and a new function to the source highlight plugin to highlight external files. The plugins section of the default server index page has been updated with usage examples.

Offline

#19 2013-01-09 18:06:36

ksira
Member
Registered: 2009-10-27
Posts: 31

Re: Xac: a markdown and Pandoc-based content management system

Looks great, I am finding more and more uses for Markdown by having Pandoc.

I've been using Hakyll, which seems to have similar functionality, to set up my blog. I would be interesting in any comparisons, beside the obvious of having to learn some Haskell (which has been rewiring my brain).

Offline

#20 2013-01-09 19:02:39

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Xac: a markdown and Pandoc-based content management system

I haven't used Hakyll so I can't offer any comparison. I take it that you've already read the default server index page for an overview of usage and available plugins. If you download the source archive you can try the default server directly and take a look at the code.

I need to sit down and provide thorough documentation when I have some time. The publishing scripts for the default and the simple server are both commented and provide good starting points for customization. The core Xac module (Xac/publish.py) may be difficult to follow in some parts but normally you shouldn't need to touch that code (that's where the markdown conversion, file interpolation and file passthrough happens). The plugins are mostly self-documenting and some of them are very simple. The code for the default server is a bit hard to follow in some places due to lack of documentation as well. It nevertheless gives you some ideas of what you can do with Xac and reminds you that it's all just Python so there are no arbitrary limitations.

If you have any questions about the code or how to do something in particular, please ask. I will try to gradually add comments to the code in response to feedback as well.

Offline

#21 2013-01-09 21:04:05

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Xac: a markdown and Pandoc-based content management system

As i was looking at the default index page I noticed that there were errors in the CDATA-like block escapes which likely caused some confusion. Sorry about that. The mistakes have been fixed in the last update.

The new file insertion plugins now support the path "%SELF%" for inserting a markdown file into itself.

Offline

#22 2013-09-14 14:58:03

Marie-Lou
Member
Registered: 2012-03-29
Posts: 6

Re: Xac: a markdown and Pandoc-based content management system

Hi,

Thank you for Xac, that's a very interesting project.

I wonder if it's possible to hide some folders in the folders list (eg. hide “etc/”).

Thanks !

Offline

#23 2013-09-14 15:36:17

Xyne
Moderator/TU
Registered: 2008-08-03
Posts: 5,688
Website

Re: Xac: a markdown and Pandoc-based content management system

Thanks for the feedback.

It's possible but a little difficult if you are using the default site configuration. The nav bar entries are determined by the get_pandoc_args function defined inside the get_default_get_pandoc_args function in Xac.defaults.py. The arguments to that function include the files and directories that will appear in the nav bar. If you define your own version then you can filter "etc/" there, but I admit the function is a bit chaotic and unpleasant to work with (the default site basically remains a hackjob). The process would be to copy that code into your own publish.py file and edit it as necessary, then pass your custom get_pandoc_args function to Xac.Xac.

If you are interested in using Xac then I recommend waiting a bit before you start hacking on it. I'm currently nearing the end of a complete rewrite that significantly improves speed, modularity and configurability*. I hope to release it soon (within the next 2 weeks) but as it progresses I keep getting ideas of how to improve it. I actually have the code open right next to the browser as I'm typing this. smile

* The rewrite also changes the implementation language (Python -> Haskell with Lua configuration files) as well as the plugin syntax, so migration will require intervention.

Offline

Board footer

Powered by FluxBB