Kingbash - menu driven auto-completion

Procyon · 2010-10-09 16:48:19

Version 27: http://aur.pastebin.com/hqh1DN8H
Quick Install: wget -O- http://aur.pastebin.com/download.php?i=hqh1DN8H | sed 's/\r$//' > kingbash; chmod +x kingbash

@shpelda: CTRL+L is \x0c, you can check with echo CTRL+V CTRL+L | hexdump

I have added CTRL+L to left as a default binding now.

Other changes:

It now uses optparse (step 1 in the big clean up). The most important change for users is that giving extra functions, aliases and builtins now needs --extracommands.

I also added --extrafiles. For now it's only usable with bash variables. It was a bit of a conundrum because you don't want those to show up until you have actually typed a $. One solution is to check if an entry in --extrafiles starts with a $. I will add that to the next version. Right now there might also be unexpected behavior with multiple variables in the same file and unrecognized ${VAR<TAB> completion.

Anyway something like echo a$BA<TAB> will complete to echo a$BASH, and that's what I was really missing.

A new example calling function for tab completion. History/dmenu works the same: (this is from the usage text)

function kingbash.fn() {
  echo -n "KingBash> $READLINE_LINE" #Where "KingBash> " looks best if it resembles your PS1, at least in length.
  OUTPUT=`kingbash --extracommands "$(compgen -ab -A function)" --extrafiles "$(compgen -v -P $)"`
  READLINE_POINT=`echo "$OUTPUT" | tail -n 1`
  READLINE_LINE=`echo "$OUTPUT" | head -n -1`
  echo -ne "\r\e[2K"; }
[[ $- =~ i ]] && bind -x '"\t":kingbash.fn'

I have not yet looked at speed improvements, but I know that typing in history mode is slow with a big history. My history file is 4.3MB and it takes about 1.5 seconds for kingbash to boot up and every key press that adds to the line also takes 1.5 seconds. The biggest problem is that the cursor can move freely so not every key press that adds to the line gives a subset of the previous list. I think the best solution is only generating a screenful, and ONLY for history mode.

EDIT:
@shpelda: I don't know what is causing that backspace error and I can't reproduce it. I mistook it for what I saw in my version for a second and didn't look further when I saw my error about an unreadable directory.

Anyway, some characters like ; & | should not let kingbash rerun and I'll add those exceptions in the next version.

Last edited by Procyon (2010-10-09 16:55:03)

Dieter@be · 2010-10-09 21:52:55

Procyon wrote:

I have not yet looked at speed improvements, but I know that typing in history mode is slow with a big history. My history file is 4.3MB and it takes about 1.5 seconds for kingbash to boot up and every key press that adds to the line also takes 1.5 seconds. The biggest problem is that the cursor can move freely so not every key press that adds to the line gives a subset of the previous list. I think the best solution is only generating a screenful, and ONLY for history mode.

yeah, stop calculating if it won't be shown on the screen anyway but why would you only optimize for history mode? dmenu mode is so similar it would benefit from the same optimisation

Procyon · 2010-10-12 02:32:12

Version 28: http://aur.pastebin.com/HQA9Hz5t
Quick Install: wget -O- http://aur.pastebin.com/download.php?i=HQA9Hz5t | sed 's/\r$//' > kingbash; chmod +x kingbash

In dvtm, which does not support turning off line wrap, it now doesn't print long headers, footers and selector bars. The environment is checked for $DVTM.

I added an option specially for URxvt users who work with double width/CJK characters. It is called --widechars. It will check lines for non-ascii and then turn line wrapping back on to avoid the freezing bug I mentioned earlier (the next urxvt release might have this fixed). It will only cause visual glitches, and only for those lines which are longer than the display and have non-ascii.

And I sped up history/dmenu mode. This speed up is ON by default. Turn it OFF with --slowhistory. It calculates only what is visible +1. Also two garbage collection tricks.

Of course, each item checked still takes the same amount of time. If you often need something near the end of a long list and have to scroll to get there, you might prefer the old method. There are no checks (yet) to see if what you added to the line gives a subset of the current list, so searching for the last item will take exactly as long.

Example: (note that `seq` takes a few seconds itself)
{ seq 1000000; echo last; } | kingbash
{ seq 1000000; echo last; } | kingbash --slowhistory
The delay between typing the letters "l" "a" "s" "t" takes equally long in both (for me about 3 seconds per letter). The speed up is only noticeable in this example when you type e.g. "9999".

If anything doesn't work quite right, let me know.

Some bugs right now:
The scroll bar is inaccurate, it can only see one page ahead. I'm not sure if this can be fixed at all.
Page down and End will not go beyond the last "known" item. This can be fixed with an offset that is added later for Page Down, and End can be set to calculate the entire list first. I'll add it to the next update.

fsckd · 2010-10-12 02:47:10

I've had this problem for all versions I've tried.

Dieter@be · 2010-10-12 11:05:21

Procyon wrote:

There are no checks (yet) to see if what you added to the line gives a subset of the current list

Isn't this true by definition? the more you add to the line, the more you restrict the current subset, no?

Procyon wrote:

Example: (note that `seq` takes a few seconds itself)
{ seq 1000000; echo last; } | kingbash
{ seq 1000000; echo last; } | kingbash --slowhistory
The delay between typing the letters "l" "a" "s" "t" takes equally long in both (for me about 3 seconds per letter). The speed up is only noticeable in this example when you type e.g. "9999".

Great. the speedup for '999' is definitely noticeable. Very cool. Although I think once having a result for "l", seeing the update for adding "a" "s" "t" should be blazing fast. (see my comment above)

Procyon wrote:

The scroll bar is inaccurate, it can only see one page ahead. I'm not sure if this can be fixed at all.
Page down and End will not go beyond the last "known" item. This can be fixed with an offset that is added later for Page Down, and End can be set to calculate the entire list first. I'll add it to the next update.

heh. only now i started looking for the scrollbar, looks cool, but is it really needed? if it is too complex to implement I would just leave it out.
For me the scrollbar, home, end, pageup, pagedown all work with this version and seem to do what I expect.

Procyon · 2010-10-12 13:27:13

@fsck: I'll add tmux next to dvtm. screen does have the line wrap off capability.

I found this site: http://bugs.python.org/file14782/test_ucs2w.py
If it's not too slow, I can use this method to count columns with double width CJK characters and make bars that go exactly to the end of the display without needing the line wrap off capability.

@Dieter@be: You can put text anywhere, so 1234 -> 12534 will not give a subset. I think a double for loop with just the `in` operator would do.

Procyon · 2010-10-12 19:24:15

Version 29: http://aur.pastebin.com/HV8xbJnd
Quick Install: wget -O- http://aur.pastebin.com/download.php?i=HV8xbJnd | sed 's/\r$//' > kingbash; chmod +x kingbash

The east_asian_width mentioned in the site above worked great and it's not slow. The filesystem entries have to be decoded though and are presumed utf8.

Now it should work without glitches/multiple line entries/crashes in anything from xterm to urxvt and multiplexers like dvtm and tmux.

The --widechars option is removed.

I added subset checking for dmenu/history. It works in --slowhistory too.

There is some complex dual lists stuff going on now (superset and original file either of which can be half way checked with scrolling). The code behind it is a bit shaky, but it seems to work right. If you notice anything either wrong or unexpectedly slow, let me know.

End will now calculate the entire list and go to the end.

I added the option --forceheight (try with --forceheight $LINES)

I'll leave Page Down and the scroll bar as is.

fsckd · 2010-10-13 03:49:21

hats off, works like a charm

Procyon · 2010-10-15 01:37:10

Version 30: http://pastebin.org/208976
Quick Install: wget -O- 'http://pastebin.org/pastebin.php?dl=208976' | sed 's/\r$//' > kingbash; chmod +x kingbash

--forceheight can now take MAX as an argument

In dmenu/history mode it now loads 10 screenfuls at once, instead of a screenful +1 line. This makes Paging Down much more comfortable, and ca. 500 items is still fast to check at once.
It is changeable with --loadpages

In the previous version there was some redundant item checking, the "shaky code" I mentioned earlier. I figured out what it needed and the code is much cleaner now. It speeds up multiple searching/browsing after having pressed End.

EDIT:
Update 188668->189948 (missing line) AUR 30-5
Update 189948->208939 ("End" going out of view after a subset search) AUR 30-6
Update 208939 -> 208976 (fixed bug in completing a command by filename right after |;&)

Last edited by Procyon (2010-10-15 16:20:55)

Procyon · 2010-10-19 20:26:41

Version 31: http://sprunge.us/KgRa
Quick Install: wget -O- 'http://sprunge.us/KgRa' > kingbash; chmod +x kingbash

Python 3 compatible.

And a bug fix in history mode where having a unique entry in the prompt caused a crash instead of returning the unique entry (like the F2 binding or like Shift+Enter in dmenu)

Update: non blocking read with no input seems to return the empty string instead of raising an exception now in python 3. This should fix using the escape key to quit.

Last edited by Procyon (2010-10-20 23:10:27)

Dieter@be · 2010-10-20 16:23:08

thanks mate.
Btw, is anyone using --slowhistory? and when/why? I never use it, even with few items in the list.

Procyon · 2010-10-31 18:17:06

Version 32: http://sprunge.us/WDAX
Quick Install: wget -O- 'http://sprunge.us/WDAX' > kingbash; chmod +x kingbash

Fixed a crash with -r FILE where FILE has non utf8. (using open()'s encoding error ignore mode)
Added 4 command line arguments
For disabling rerunning kingbash (I recommend it with backspace):
--noretab-az - regular keys
--noretab-backspace - (alt+) backspace
--noretab-directory - left and right arrow keys
And to make * act as globbing and move its old use of adding all possible completions to the prompt to +
--plustoall

* is added escaped (noticeable in the prompt and after pressing Escape), but is still interpreted as globbing in kingbash. It's useful in a situation where you want to reduce the list of completions with something at the end of the filename, for example cat *.txt[TAB] can now be done as cat [TAB]*.txt

Dieter@be · 2010-10-31 18:44:58

I noticed a bug btw with history searching(with v31): if you load a history entry, then press some random keys (so there are no search matches), then tab, it crashes (i wish tab would do file completion or bash-like completion, but just not crashing would be fine too)

Procyon · 2010-10-31 20:17:13

Version 33: http://sprunge.us/AWTX
Quick Install: wget -O- 'http://sprunge.us/AWTX' > kingbash; chmod +x kingbash

@Dieter@be: That could indeed be very useful.

Normally tab and insert will copy the selection to the current edited line, just like tab in dmenu.

I added the command line argument
--historytabcompletion

In dmenu/history mode you can then press TAB and it will be as if you did TAB completion on that line and cursor position. (including --plustoall and --noretab-backspace)

Cloudef · 2010-10-31 20:32:10

Thank you, works wonders. Really like the idea of this also.

Dieter@be · 2010-10-31 21:46:13

Procyon wrote:

You can change the calling function too for this. For instance
... < sort -u ~/.bash_history
Or preserving the reverse order where the most recent item comes first:
... < cat -n ~/.bash_history | sort -u -k 2 | sort -nr | cut -f2-

I finally figured out what's wrong with this last line. To achieve uniqueness for key 2, the 2nd step in the pipe will take the first occurence of key 2, i.e. the earliest call. The outcome is that ultimately, when you see your "dmenu-like" list you usually don't see the most recent call at the top. Instead commands are ordered by the first invocation found in the history file.
I now replaced it with this:

... < tac $HISTFILE | cat -n - | sort -u -k 2 | sort -n | cut -f2-

Procyon · 2010-10-31 23:52:11

@Dieter@be: Nice catch. I added the same functionality with a command line switch, but it's slower in pure python by about 1.5x with some algorithms I found with google.
I also removed the code that inhibited two identical items next to each other.

@Cloudef: Thanks and yeah bind -x allows nicely for well-integrated command line helpers (even mouse support - I am working on a better version of that.). For kingbash as long as it can continue without a configuration or session file I'll add anything.

crnibojan · 2010-11-02 08:04:43

First and foremost: great job! I like this tool a lot.

There is one small annoyance about it, which bothers me at the moment. Namely, I use multiline prompt, looking something like this:

PS1="[\h] @\t [\w]\n\u> "

kingbash overwrites only second line of the prompt. As prompt is printed again after kingbash exits, it causes that first line of prompt is printed and visible as many times as auto completion is tried in same line. Is there / will there be cure for this?

crnibojan · 2010-11-02 08:21:23

And one more thing: kingbash and shell variable CDPATH do not go well together.

I have something like this in my .bashrc:

export CDPATH=.:~:/media/work/repositories:

Without kingbash, shell will auto-complete, for cd command, any folder under /media/work/repositories from anywhere. With kingbash that doesn't work.

crnibojan · 2010-11-02 08:56:20

crnibojan wrote:

And one more thing: kingbash and shell variable CDPATH do not go well together.

Actually, problem is more generic: kingbash kills auto-completion for command options. cd is not best example to illustrate the problem (although, it looks like it will be most difficult to solve, as CDPATH makes cd check several locations, while kingbash checks only current folder). For instance, you can get list of git options starting with 'c' by typing:

> git c<TAB><TAB>
checkout cherry cherry-pick clean clone commit config

When kingbash is bind to TAB, this does not work any more. My workaround, for time being, is to bind kingbash to some other key.

Procyon · 2010-11-02 14:52:48

1.
There is a simple way to fix the PS1 issue and that is by changing the last echo in the calling function.

bind -x clears only the current line. In the case of a multi-line PS1 or several lines of text in the prompt then only the last line is cleared.

The PS1 problem is easy to fix by changing
echo -ne "\r\e[2K"
to
echo -ne "\r\e[2K\e[A\r\e[2K"
i.e. after Kingbash is finished, clear the current line (things like one or zero possibility completions don't write anything), go one line above and clear everything there.

You can also change the first echo to move up (so Kingbash overwrites the first line of PS1) but then the text in the prompt will move a lot.

A long line in the prompt is a bit trickier and needs editing the PS1 with a Cursor Save escape code. The first echo in the calling function can then issue a Cursor Restore escape code.

I don't want to force users to change the PS1 for just a visual glitch though.

2.
I will look into this.

3.
bash-completion integration would be great. I haven't experimented a lot with it though. I don't know of a way to "run a line by it" for example.

If that is impossible, for command line options for commands there are some options:
Rewriting completions would be wrong.
Running and interpreting `git help` or `command --help` isn't very wise.
Interpreting /etc/bash-completion/command might be possible since most of them have a "compgen -W" with a list of command line options.

It might also be possible to make bash-completion run kingbash in either dmenu mode for users/groups/kernel versions/cmd options/etc or regular mode for files/directories.

Procyon · 2010-11-02 19:33:17

Version 34: http://sprunge.us/ZLLZ
Quick Install: wget -O- 'http://sprunge.us/ZLLZ' > kingbash; chmod +x kingbash

Code changes:
diff -Naur <(curl -s http://sprunge.us/AWTX) <( curl -s http://sprunge.us/ZLLZ)

This has the python-only sorting method and in the comments is a calling function that does what I mentioned above about long text in the prompt and it also counts PS1 for \n and removes that many lines at the end. I'll wait with putting this on AUR until next version.

Here is the alternative calling function.

PS1+='\[\e[s\]'

function kingbash.fn() {
  local KBCOUNT OUTPUT KBMOVE KBLINES
  echo -ne '\e[u\e[6n'
  read -sdR
  [[ $REPLY =~ ..([0-9]*).([0-9]*) ]]
  (( ${BASH_REMATCH[1]} == LINES )) && KBMOVE=A || KBMOVE=B
  KBLINES=$(((${BASH_REMATCH[2]} + ${#READLINE_LINE} ) / COLUMNS))
  for ((KBCOUNT=0;KBCOUNT < $KBLINES; KBCOUNT++)); do
    echo -ne '\r\e[2K\e['$KBMOVE
  done
  if [[ $KBMOVE == B ]]; then
    echo -ne "\e[$KBLINES"A
  fi
  echo -ne "${READLINE_LINE:0:$((COLUMNS-${BASH_REMATCH[2]}))}"
  OUTPUT=$(kingbash --plustoall --noretab-backspace --extracommands "$(compgen -ab -A function)" --extrafiles "$(compgen -v -P $)")
  READLINE_POINT=$(echo "$OUTPUT"|tail -n1)
  READLINE_LINE=$(echo "$OUTPUT"|head -n -1)
  echo -ne '\r\e[2K'
  [[ $PS1 =~ \\n ]]
  for (( KBCOUNT=0; KBCOUNT < ${#BASH_REMATCH}; KBCOUNT++ )); do
    echo -ne '\e[A\r\e[2K'
  done
}
bind -x '"\t":kingbash.fn'

Procyon · 2010-11-03 11:36:23

I forgot to mention that I also added CDPATH checking. It works really simple right now. Basically if the command is cd, then all items in the paths of CDPATH are added. Because of 'recommended' highlighting, all the directories will be on top in different color. The path will be added in full, which also gives different behavior. You won't cd to usr, but to /usr, and cd won't print the destination like it does with a CDPATH substitution. On the other hand the non-dependence of cd's CDPATH substitution gives some new features as well that are also useful for other commands (i.e. access CDPATH with `cd TAB` and pressing F2 on an item you would like to work with).

I'll upload it to AUR now.

Procyon · 2010-11-07 23:05:48

Version 35: http://sprunge.us/NgUP
Quick Install: wget -O- 'http://sprunge.us/NgUP' > kingbash; chmod +x kingbash

Code changes:
diff -Naur <(curl -s http://sprunge.us/ZLLZ) <( curl -s http://sprunge.us/NgUP)

This version adds --bashcompletion for reading arguments of commands in /etc/bash_completion.d/COMMAND

This will run the actual completion function that is inside the file, trying to make the environment the same as when it is called by bash. None of the completions in /etc/bash_completion are added, like username/kernel version/etc. The file /etc/bash_completion is sourced to provide usability functions for the command completions files. However, one of the most important ones didn't work and I wrote a substitution. I made it fit the expectations of all the command files I had, which was just filling in "$cur" and "$prev". This could break some other scripts. I also didn't experiment with all the files in /etc/bash_completion.d/ and I expect other functions will have to be rewritten. If you find a command generating errors or no completions, let me know and I'll improve it a bit.

Also fixed a bug in relaying command line switches to the next process when rerunning kingbash.

Count tabs in the right way.

Update ebFT->NgUP: made tab neater and corrected typo in help text.

Last edited by Procyon (2010-11-07 23:33:20)

crnibojan · 2010-11-08 15:19:18

Procyon wrote:

Version 35: http://sprunge.us/NgUP

I hit few problems with this version.

1) I'm trying to execute this command:

. .bashrc

If I type:

. <TAB>

following is printed out:

bash: line 3: source: /etc/bash_completion.d/.: is a directory
bash: line 6: _.: command not found
KingBash> .

2) Pressing TAB on empty line (rather pointless, I know, but it happens to me rather often) causes following error:

Traceback (most recent call last):
File "/home/radakovi/bin/kingbash", line 1101, in <module>
main()
File "/home/radakovi/bin/kingbash", line 335, in main
if os.path.exists("/etc/bash_completion.d/"+lastcommand[0]):

Same error is visible if, for instance, one does this:

ls <TAB><BACKSPACE><BACKSPACE><BACKSPACE>

Adding --noretab-backspace as option to kingbash call solves problem above, but I don't want to do that, as I like default behavior

Arch Linux

#76 2010-10-09 16:48:19

Re: Kingbash - menu driven auto-completion

#77 2010-10-09 21:52:55

Re: Kingbash - menu driven auto-completion

#78 2010-10-12 02:32:12

Re: Kingbash - menu driven auto-completion

#79 2010-10-12 02:47:10

Re: Kingbash - menu driven auto-completion

#80 2010-10-12 11:05:21

Re: Kingbash - menu driven auto-completion

#81 2010-10-12 13:27:13

Re: Kingbash - menu driven auto-completion

#82 2010-10-12 19:24:15

Re: Kingbash - menu driven auto-completion

#83 2010-10-13 03:49:21

Re: Kingbash - menu driven auto-completion

#84 2010-10-15 01:37:10

Re: Kingbash - menu driven auto-completion

#85 2010-10-19 20:26:41

Re: Kingbash - menu driven auto-completion

#86 2010-10-20 16:23:08

Re: Kingbash - menu driven auto-completion

#87 2010-10-31 18:17:06

Re: Kingbash - menu driven auto-completion

#88 2010-10-31 18:44:58

Re: Kingbash - menu driven auto-completion

#89 2010-10-31 20:17:13

Re: Kingbash - menu driven auto-completion

#90 2010-10-31 20:32:10

Re: Kingbash - menu driven auto-completion

#91 2010-10-31 21:46:13

Re: Kingbash - menu driven auto-completion

#92 2010-10-31 23:52:11

Re: Kingbash - menu driven auto-completion

#93 2010-11-02 08:04:43

Re: Kingbash - menu driven auto-completion

#94 2010-11-02 08:21:23

Re: Kingbash - menu driven auto-completion

#95 2010-11-02 08:56:20

Re: Kingbash - menu driven auto-completion

#96 2010-11-02 14:52:48

Re: Kingbash - menu driven auto-completion

#97 2010-11-02 19:33:17

Re: Kingbash - menu driven auto-completion

#98 2010-11-03 11:36:23

Re: Kingbash - menu driven auto-completion

#99 2010-11-07 23:05:48

Re: Kingbash - menu driven auto-completion

#100 2010-11-08 15:19:18

Re: Kingbash - menu driven auto-completion

Board footer