lapr - a commandline IDE for LaTeX

hyde · 2016-04-25 13:15:32

lapr - a commandline IDE for LaTeX

lapr is a tool to help writing documents with LaTeX. Its main features are automatic code generation (e.g. packages in the preamble, environments), project management and helping the user to concentrate on content, not layout/markup.

Dependencies

lua >= 5.2
lua-penlight

Development Status

Very early stage. I have implemented the basic functionality and libraries, but there will still be many changes in the interface. I just wanted to show this here to get some feedback and maybe some interest. Also, i'm curious what you're missing in your LaTeX-flow.

Idea and Reason for this program

I write alot of LaTeX documents and i'm doing this on the commandline using vim. I don't like to see all the files latex creates during compilation and i hate the output of the latex compiler. In the past, i used a simple makefile to hide all files after compilation and fetch them, if necessary (for example the .aux file). I didn't do anything about the output. Furthermore, i used templates to reduce the amount of code you have to write, but i still have the feeling i'm writing too much code by hand.

What i needed was a tool to help me with this problems. The idea was to have something similar to a shell, a commanddriven latex project manager. You should be able to create projects, add content to them, compile, view the output, edit etc. This should be configurable so that everybody can use their favourite editor just by saying 'edit'.
Furthermore, i want make writing of latex documents more easy by using code generators. For example, i don't know any package except the tikz package which uses the 'tikzpicture' environment. Why not insert the package when a tikzpicture is encoutered?
I also like the content structuring methods of LyX, how you can generate new chapters and easily move them from one place to another. This is (for me) a common task, why do this in the editor?

Well, there is a lot more stuff like this, but i'm currently only implementing basic stuff. I started on simple package detection and it works quite good, but it's not finished yet.

Usage

After installing, the program is available with the command 'lapr'. No arguments are needed, you can look at the currently supported by lapr -h or --help.
When you get the prompt, you can get a list of all commands with 'help'. Every command has a small description with 'help command'.
The basic workflow is:

create document
add content introduction
edit introdocution
compile
view

Don't expect too much. I want to process the logfile to get good warnings and errormessages, but there is nearly nothing there. Try simple latex stuff like \kant or something. Try to use a command from a package. If it is foreign (most likely) for the program database, you will be asked for the package and it will be added to your preamble. If something is a latex-command like \cleardoublepage then name the package 'latex' (exactly written like this).

Goals

Parser for compiler output
automatically fix errors or jump to the write location
implement further code generation for sections, chapters etc.
make everything configurable

and much more, this is a small roadmap.

For you

Here is a simple PKGBUILD to try it out:
PKGBUILD has changed on 01.05.2016 (first of may)

# Maintainer: Patrick Schulz <pschulz at posteo dot de>
pkgname=lapr-git
pkgver=0.1
pkgrel=1
pkgdesc="A commandline IDE for LaTeX documents"
arch=('i686' 'x86_64')
url="https://github.com/patrickschulz/lapr"
license=('GPL')
source=('git+https://github.com/patrickschulz/lapr.git')
makedepends=('git')
depends=('lua' 'lua-penlight')
md5sums=('SKIP')

build() {
    # temporarily, since i'm working on the branch 'dev'
    cd ${srcdir}/lapr
    git checkout dev
}

package() {
    mkdir -p ${pkgdir}/usr/share/lua/5.3/lapr/data
    mkdir -p ${pkgdir}/usr/bin
    cp -R ${srcdir}/lapr/laprlib ${pkgdir}/usr/share/lua/5.3/
    cp ${srcdir}/lapr/main.lua ${pkgdir}/usr/bin/lapr
    cp ${srcdir}/lapr/data/packagelookup.lua ${pkgdir}/usr/share/lua/5.3/lapr/data
    cp ${srcdir}/lapr/data/templates.lua ${pkgdir}/usr/share/lua/5.3/lapr/data
    cp ${srcdir}/lapr/config.lua ${pkgdir}/usr/share/lua/5.3/lapr
    chmod 755 ${pkgdir}/usr/bin/lapr
}

If you're not using lua5.3, change the lines accordingly to your version.
The actual version uses lua-readline, but this is in luarocks, not in AUR. I just changed the lines in laprlib/session.lua. The lines are commented out, you can bring them back if you want to. It works fine without.

I just wanted to release this to see what you think of it. I will be happy for suggestions, ideas and feature requests! Thank you very much.

Last edited by hyde (2016-05-01 10:25:44)

stupidus · 2016-04-27 09:49:23

Interesting project.

You seem to have some hardcoded paths to your own home directory in your code. I got the following warning when running lapr:

could not read system package database ('/home/pschulz/Workspace/lua/lapr/data/packagelookup.lua')

Then there was also a warning about the user package database.

Also, how can you continue working on a previous project? The load command does not work (missing .project). Or do you need to explicitly call the save command before quitting lapr?

hyde · 2016-04-27 12:02:42

Yes, you are right. I changed the paths already to

getenv("HOME") .. "/.config/lapr/data"

but i still have to make the push to github. I'm at work right now but i will make it tonight (so in about 4-6 hours).
The warning you are getting is right, it says that you have no user defined packages. It should be a debug message, it will be in future.

To work on a existing project you have to save your project explicitly. I will add a hook so it autosaves on exit (as a commandline switch).

hyde · 2016-04-27 14:25:56

I fixed the issue with the paths, there are two files which will be loaded: the system database and the user database.
After installing you have no user database. If you use commands unknown to the system database you will be asked for a package. After closing the program, the new packages will be saved in your home config directory. You can also use this file to overwrite the system preferences.

Also, autosaving is now available, but currently not as default. Use

lapr -a

to enable autosaving. After this, a simple

lapr

in the current directory should bring you back your project. Check this with

info

shmibs · 2016-04-28 04:16:34

don't have much to add here other than "support!" and "encouragement!", but this seems like it could definitely be a useful, useful thing ^_^

stupidus · 2016-04-29 18:14:52

With your newest version I get the following error, when entering "create document". I figured, that it is related to the missing config file in ~/.config/lapr/, so I created an empty config file. Now it does not crash anymore.

stack traceback:
	[C]: in function 'dofile'
	/usr/share/lua/5.3/laprlib/project.lua:116: in method 'load_config_file'
	/usr/share/lua/5.3/laprlib/project.lua:87: in function 'laprlib.project.create'
	(...tail calls...)
	/usr/share/lua/5.3/laprlib/session.lua:543: in method 'execute_command'
	/usr/share/lua/5.3/laprlib/session.lua:794: in method 'loop'
	/usr/bin/lapr:165: in main chunk
	[C]: in ?

Is there any documentation on the config file yet? I suppose it has to be written in lua, right? Could you maybe provide a example config file?
Is it also possible to change the default template via the config file? The current one seems to be hardcoded using KOMA script and german language.
EDIT: I am stupid. When I was looking at the source code on github, I did not realize that I was looking at the master branch instead of the dev branch. I found the config file now.

I also have a sort of feature request:
Do you think it makes sense to have the ability to have multiple default templates, i.e. one for articles, one for beamer etc.?
As in "create document {article,beamer,book,foo}"?

EDIT: Now I get an error on "compile"

document: 4 > compile
lua: /usr/share/lua/5.3/pl/stringx.lua:27: argument 1 expected a 'string', got a 'nil'
stack traceback:
	[C]: in function 'error'
	/usr/share/lua/5.3/pl/utils.lua:388: in function 'pl.utils.assert_arg'
	/usr/share/lua/5.3/pl/stringx.lua:27: in upvalue 'assert_string'
	/usr/share/lua/5.3/pl/stringx.lua:419: in function 'pl.stringx.lines'
	/usr/share/lua/5.3/laprlib/latex.lua:30: in function 'laprlib.latex.parse_output'
	/usr/share/lua/5.3/laprlib/project.lua:387: in function </usr/share/lua/5.3/laprlib/project.lua:383>
	(...tail calls...)
	/usr/share/lua/5.3/laprlib/session.lua:543: in method 'execute_command'
	/usr/share/lua/5.3/laprlib/session.lua:794: in method 'loop'
	/usr/bin/lapr:165: in main chunk
	[C]: in ?

Last edited by stupidus (2016-04-29 18:23:35)

hyde · 2016-05-01 08:55:47

With your newest version I get the following error, when entering "create document". I figured, that it is related to the missing config file in ~/.config/lapr/, so I created an empty config file. Now it does not crash anymore.
stack traceback:
[C]: in function 'dofile'
/usr/share/lua/5.3/laprlib/project.lua:116: in method 'load_config_file'
/usr/share/lua/5.3/laprlib/project.lua:87: in function 'laprlib.project.create'
(...tail calls...)
/usr/share/lua/5.3/laprlib/session.lua:543: in method 'execute_command'
/usr/share/lua/5.3/laprlib/session.lua:794: in method 'loop'
/usr/bin/lapr:165: in main chunk
[C]: in ?
Is there any documentation on the config file yet? I suppose it has to be written in lua, right? Could you maybe provide a example config file?
Is it also possible to change the default template via the config file? The current one seems to be hardcoded using KOMA script and german language.
EDIT: I am stupid. When I was looking at the source code on github, I did not realize that I was looking at the master branch instead of the dev branch. I found the config file now.

Yes, i will include a sample config file with all possible values in my next commit. Also, you are right, the file has to be written in lua. As you can see, the file must return a table, since it will be loaded by a load(). I want to make this more flexible by making the 'return {' optionally, but since this is not so important i haven't done it yet.

I also have a sort of feature request:
Do you think it makes sense to have the ability to have multiple default templates, i.e. one for articles, one for beamer etc.?
As in "create document {article,beamer,book,foo}"?

Oh yeah, of course! Without this, a project management wouldn't be very flexible. I will try to implement this feature in the next days, then you can try it out.

Concerning the error while compiling: i'm on it, i just found some more errors. I will fix them and make a commit, i think today.
EDIT: Fixed the compile issue, was a stupid swap of two lines

Thank you for testing

EDIT2: I just made a push, so you can try out the new version. I updated the PKGBUILD in my first post, the sample configuration file gets now copied to /usr/share/lua/5.3/lapr/config.lua. You can have a look there and copy it to $HOME/.config/lapr/config.lua.

Last edited by hyde (2016-05-01 10:23:31)

hyde · 2016-05-01 10:26:28

shmibs wrote:

don't have much to add here other than "support!" and "encouragement!", but this seems like it could definitely be a useful, useful thing ^_^

Thank you very much! If you have any ideas or feature requests, let me know.

hyde · 2016-05-01 10:29:56

Regarding documentclass templates: You can now add an argument to create (e.g. create document scrartcl) for the documentclass to be used. Unfortunately, this is currently all it does. There is no preamble template to be inserted, just the class itself. You will have to edit the preamble yourself, but that is unfortunatly currently not possible since it will be generated automatically.
I will introduce a mechanism, which detects manual changes to the preamble and merges them with the generated code. But ideally, you shouldn't have to worry about writing the preamble at all, so, of course, i will add some kind of template mechanism.
Stay tuned

stupidus · 2016-05-01 19:59:05

Thanks a lot for your effort. The compiling and autosaving is working now.
I copied the config.lua file, and tried to change the compiler. I tested it with both latexmk and pdflatex (latexmk is configured to use pdflatex anyway on my system). However, according to the log file it is still using luatex to compile. Also, I tested to change the editor to nano, and the pdfviewer to qpdfview. None of those settings is taken into account.

hyde · 2016-05-01 20:44:01

stupidus wrote:

Thanks a lot for your effort. The compiling and autosaving is working now.
I copied the config.lua file, and tried to change the compiler. I tested it with both latexmk and pdflatex (latexmk is configured to use pdflatex anyway on my system). However, according to the log file it is still using luatex to compile. Also, I tested to change the editor to nano, and the pdfviewer to qpdfview. None of those settings is taken into account.

Hm, that is strange. I can't reproduce this right now, what is your exact setup? I did two more commits right now, but they don't have anything todo with the handling of the configuration file. Maybe i did forget a push before that.
However, this is my session, as you can see, it works for me. Try it out, but ideally get the newest state of development, to be sure.

lapr: 1 > create doc
could not load user templates file (perhaps you don't have one)
doc: 2 > add content test
doc: 3 > set editor
nano
doc: 4 > edit
doc: 5 > set engine
pdflatex
doc: 6 > set raw true
doc: 7 > compile
This is pdfTeX, Version 3.14159265-2.6-1.40.16 (TeX Live 2015/Arch Linux) (preloaded format=pdflatex 2015.11.15)  1 MAY 2016 22:37
entering extended mode
 restricted \write18 enabled.
...

The config file i'm using lays in $HOME/.config/lapr/config.lua and contains (this is the content I uploaded):

return {
    -- editing, viewing and compiling environment
    engine = { exe = "pdflatex", args = "" },
    viewer = "evince",
    editor = "nano",

    settings = {
        --ignore_hidden = false, -- used for deletion of files
        raw_output = false, -- parse LaTeX output (false) or display LaTeX output plain (true)
        --debug = false,
    },

    -- file names
    --[[
    file_list = {
        main_file = "main",
        project_file = ".project",
        preamble_file = "preamble"
    },
    --]]

    -- auxiliary files. This list is used to protect files not belonging to the project while cleaning up
    --aux_files = { },

    -- directories in which files will be stored
    --[[
    directories = { 
        file_dir = "files",
        image_dir = "images",
        project_dir = ".build",
        minimal = ".minimal",
    },
    --]]

    -- used for diffs to compare edits
    --temporary_file = ".difftempfile",
}

Important: when you create a project, you should get the message about a missing templates file. If the configuration file could not be loaded, you should get this message as well. Then there is something wrong. So your first lines should NOT look like this:

lapr: 1 > create doc
could not load user config file (perhaps you don't have one)
could not load user templates file (perhaps you don't have one)

I'm sorry, i'm currently not sure why this is happening. I will have a look at this. But thank you very much for keeping up with the unfinished, erroneous state lapr is.

Last edited by hyde (2016-05-01 20:44:40)

stupidus · 2016-05-02 08:20:56

Ok, I understand it now. When you create a project, it writes the current configuration into the .project file, so any subsequent changes to ~/.config/lapr/config.lua are ignored. Is this the intended behaviour?

hyde · 2016-05-02 09:07:39

Oh, ok. I would not have thought of that. Thank you. This is not intended. But maybe it would be nice to have an option for this. Then you could choose if the config file overwrites the project file or vice-versa.
I haven't used the load function much, so i'm not sure, but the fix should be easy, you can try it out, if you want to edit the files:
This is the code of the load function: (line 329 in laprlib/project.lua in the current state)

function meta.load()
    local project_table = pl.file.read(".project")
    if not project_table then
        print("couldn't read project file ('.project')")
        return nil
    end
    local self = pl.pretty.read(project_table)
    setmetatable(self, meta)
    return self
end

You have to insert

self:load_minimal_templates()

after the loading of the project table, before the return (obiously). The setmetatable shouldn't bother. It has to be placed after the metatable line.
I will test this and make a commit tonight.

EDIT: I made the commit, it works. Not choosing, though. I think this is sensible, but this requires a different design of settings handling. I am working on this anyway since it is not very sensible to have to create a project in order to set the editor or the engine.

Last edited by hyde (2016-05-02 20:32:59)

stupidus · 2016-05-03 11:44:32

I can confirm, that it is working correctly now. I tested different editors and different compilers. I think once I saw some message, that latexmk is not yet supported, but I could not reproduce it, and it is actually working fine with latexmk.

Arch Linux

#1 2016-04-25 13:15:32