NEXTVI
======
Nextvi is a vi/ex editor. It can edit bidirectional UTF-8 text.
NOTICE
======
As of 2022-01-01 Nextvi development has been driven to completion.
Be sure to read Q1 in the FAQ section of this file before doing anything else.
NAME
======
Nextvi tends to favor clean implementations over 100% adhering to POSIX like
neatvi does. Thus the name "next", to signify that we are thinking outside
the box. The notable changes that I am referring to (see below): 56, 60, 61.
Some keybinds specified by POSIX were not implemented in original neatvi,
nextvi might use them for some features below.
NEATVI FEATURES
---------------
- ex options:
can be set using :se option or :se option=value
can be unset using :se nooption
td
Current direction context. The following values are meaningful:
* +2: always left-to-right.
* +1: follow conf.c's dircontexts[]; left-to-right for others.
* -1: follow conf.c's dircontexts[]; right-to-left for others.
* -2: always right-to-left.
shape
If set (default), performs Arabic/Farsi letter shaping.
order
If set, reorder characters based on the rules defined
in conf.c.
hl
If set (default), text will be highlighted based on syntax
highlighting rules in conf.c.
hll
If set, highlight current line.
ai
As in vi(1).
aw
As in vi(1).
ic
As in vi(1).
- special marks:
* the position of the previous change
[ the first line of the previous change
] the last line of the previous change
- special yank buffers:
/ the previous search keyword
: the previous ex command
- ex commands:
:cm[ap][!] [kmap]
Without kmap, prints the current keymap name. When kmap is specified, sets
the alternate keymap to kmap and, unless ! is given, switches to this keymap.
:ft [filetype]
Without filetype, prints the current file type. When filetype is specified,
sets the file type of the current ex buffer.
In nextvi :ft also reloads the highlight ft, which makes it possible to reset
dynamic highlights created by options like "hlw".
- new key mappings (normal):
^a searches for the word under the cursor.
zL, zl, zr, and zR change the value of td option.
ze and zf switch to the English and alternate keymap.
gu, gU, and g~ switch character case.
^l updates terminal dimensions and redraws the screen.
- new key mappings (insert):
^p inserts the contents of the default yank buffer.
^e and ^f switch to the English and alternate keymap.
*vi(1) - for features unspecified refer to the respective page in the POSIX
manual.
NEXTVI FEATURES & CHANGES
-------------------------
1. Added unindent keybind: ^w
^w may also take vi_arg1 or motions as a region.
2. Added quickexit to insert mode keybind: kj
It's important to understand that this has a caveat, if input comes too fast,
for example a paste from terminal and the sequence has "kj" in it, insert
mode will be cancelled mid way and paste will be wrong. Luckily this almost
never happens because such letter combination is extremely rare.
Can be disabled via :se noqe
3. Added linenumbers, keybind: # (relative numbers are placed after indent)
keybind: v# enables permanent line numbers
4. Added invisible character view, keybind: V
5. Added regex for changing spaces to tabs, keybind: vi
6. Added regex for removing /r in dos files and trail space/tab, keybind: vo
7. Changed behavior of ^a to change search direction when no more match. (see
49.)
8. Added fssearch, searches what is under the cursor or (last search kwd)
in every file in the opened directory keybind: ^] or ^5
By default it checks every file, else filter is specified by :inc (see 32.)
This works in compliance with other changes (see 49. 58.)
If max number of available buffers reached, the last buffer will be reloaded;
losing potential unsaved changes. Use fssearch for code navigation between
files,
jump to definitions / etc. fssearch runs 40% faster than busybox's grep,
benchmark tested (time to find something that doesn't exist) on the linux
kernel.
9. fssearch but going in reverse keybind: ^p
10. Added key to change the mark origin which ^p will return to, keybind: ^t
11. Added ex command "ea" which opens file ignoring the path,
setting an extra parameter in form of a number will tell how many matches with
similar filename to skip.
For example file might be named "./path/bla/bla/file123.c" but you can open it
just by "ea fi" if this is a uniquely matched filename.
12. Added ex command "fd" to set the directory for fssearch or "ea" ex command
Running "fd" without an arg will recalculate directory info at current working
dir.
13. Added numbered buffers to vi, default 10 and ex "b" command to show buffers
and "b%d" to switch (where %d is the buffer number).
Added new ex command "bx", where the argument will change the number of buffers
allowed. If the number is lower than number of buffers currently in use they
will be deallocated. Running bx without an arg will reset to default value of
10.
Increasing the number of buffers may result in a positive effect on performance
of
fssearch (8. 9.) if the search keyword does not change. This is because vi
will remember the position of previous match in the buffer avoiding redundant
search.
Improved buffer pathname expansion shortcuts. If you use character % or # in
ex prompt they will substitute the buffer pathname. % substitutes current
buffer and # last swapped buffer. Now it is possible to expand any arbitrary
buffer by using % or # (no difference in this case) followed by the buffer
number. Example: :!echo "%69" prints the pathname for buffer 69 (if it exists).
% and # can be escaped normally if path expansion is not wanted.
14. Added key to show buffers and switch buffer
(to switch press corresponding 0-9 number) keybind: ^7
If vi_arg1 is specified right before ^7 the buffer will be switched immediately
which also happens to permit numbers > 9.
15. Added key to exit vi (not saving changes) keybind: qq or zz
16. Added key to goto line 0 keybind: gg
17. Added key to delete everything inside (cursor outside) "" keybind: di" or
dc"
18. Added key to delete everything inside () keybind: di) or dc)
19. Added key to delete everything inside (cursor outside) () keybind: di( or
dc(
20. Added key to change . to -> on cursor line keybind: vh
21. Added key to change -> to . on cursor line keybind: vg
22. Added new ex option "ish", this makes every "!" pipe command
run though interactive shell so that stuff like aliases work. By
default it is enabled, can be disabled if you need to for some
odd reason via :se noish
23. Changed the colors to be based on standard ANSI 16 colors desc in conf.c
Colors can be customized up to 256 colors if the terminal supports it.
24. Added new syntax highlighting for C, js, html, css, diff...
25. Added key that splits the line (opposite of J) keybind: K
K may also take vi_arg1, in which case it will repeat the operation on next
line.
26. Added key that line wraps text to 80 line limit keybind: gq
keybind: gw does the same but floors to the last word and is also
much slower. gw is a macro, so set td=2 if there is bidi text, it makes a
difference.
27. Added key that does multiline repeated edits keybind: v.
This is based on the last commands and insertions and requires vi_arg1 for
how many lines to repeat said operation. When last operation was 'i' or other
commands that enter insert mode and some text, that text will be placed at
the same offset on N number of lines specified by the vi_arg1.
28. Added ability to view the numbers for arguments that keys e,w,E,W,b,B
may take. keybind: ^v Pressing again will change the key mode, specifying
any vi_arg1 will exit the mode. This is a major step up to how navigation
works in vi, it makes it so much easier to use because now you can see where
you are going. These special numbers and their colors can be customized
through conf.c. As of the latest git version, the feature will work correctly
even if there is bidi text, double width characters and text reordering.
29. Added ability to change highlight dynamically. (via syn_reloadft();)
30. New ex option "hlw" which highlights every instance of word on the
screen based on cursor position. Useful for when studying source code.
31. Added autocomplete in insert mode. Press ^g to index the current opened
file. Then you can press ^n to cycle though the options, results are based on
the contents of the file and the closest match to what you typed. Use ^r to
cycle in reverse.
By default, it will use big regex like [^;...]* to sort out all the punctuation
chars from words and build a database of words. But in order to take full
advantage out of the completion system, you can change this regex at runtime
using new ex command "ac". For example say we don't want word completion
and we want entire line completion instead, run :ac .*
If the regex rule allows inclusion of nonalphanumeric or punctuation characters
you won't be able to retrieve the string like it works with words by default.
Automatically determining the position from which completion starts
inside insert sbuf is ambiguous. More fine tuned control is needed and
can be achieved using new keybind ^z in insert. Use ^z before you type out
search term, this will set start position for completion, such
that the options can be looped over inplace. ^z is a toggle, to disable
it (without exiting insert) press twice in same place. When ^z is set
^u keybind will delete everything until ^z mark first, otherwise ^u
operates normally (deletes everything).
Autocomplete db is persistent throughout all buffers and it also has data
duplication and redundancy checking such that same files can be indexed many
times.
Like ^g used to index file in insert mode, ^y can be used from insert to
clear out the completion db.
Running ex command "ac" with no argument will reset back to the default
word filtering regex. You can find it's string in led.c as a reference.
Using ^b from insert mode will display all possible autocomplete options.
Added new ex option "pac". When enabled the autocomplete options will
be automatically displayed.
32. Added ex command "inc" which sets the path filter using regex.
Example 1:
We want to get only files in submodule directory that end with .c extension:
:inc submodule.*\.c$
Example 2:
Exclude the .git folder filled with useless clutter.
:inc (^[^&&.git]+[^\/]+$)\\|(<optional branch for exceptions here>)
Running "inc" without an arg will disable all filters.
33. Added file manager keybind: \
Commands listed below under (33.) will work everywhere in vi.
- New ex command "cd" changes working directory according to 1st arg.
- (32.), (12.) and ex "cd" affect how/what dirs & files are listed.
- Added key that opens the file based on text from the cursor.
keybind: ^i or TAB
- when inside /fm/ using keybind \ again will dynamically update the
directory info at current working dir.
34. Added key to save current file keybind: ^k
35. The new special character "/" for ft in conf.c now signifies that the
regex will be applied on any file. "/" is forbidden filename character on
unix, the filetype that includes "/" is for internal use.
36. Added a window size signal handler for vi to redraw the screen
automatically.
37. Added history buffer for ex commands when in vi prompt.
keybind: ^b or vb (from normal) will open the buffer with all previous
commands move the cursor to wanted command and exit buffer with qq
will copy the command into prompt. To immediately execute the command exit
with zz. You can also use this when half-way through some command and need to
access normal mode to edit the command more efficiently. The filename is
named "/hist/" so that it can't be written to file with ^k (by accident). To
save history to file use :w yourfilename
38. Added key to grab the current word(s) under the cursor into prompt like
so :%s/.../ keybind: vr (see also 49.)
39. Added ex option to change number of spaces in a tab (\t) default is set
to 8. use :se tbs=N (N is number of spaces)
40. Added partial support for multiline block regex, for example C multiline
comments syntax highlight.
41. Made the lowest row to not waste any space when there are no messages and
actually display the row.
42. Search via '/' or '?' automatically centers and redraws screen. This
partly because change in 41 makes the bottom row behave dynamically and you
might get search result on displayed on that row, which will be covered by
search message instead. Also, centering is nice because you always know
where to expect the result to be with your eyes.
43. Added terminal clean up on exit
44. Added a key to perform relative word replacements keybind: vt
Specify vi_arg1 and the word(s) under the cursor will be placed into prompt,
for example :.,.+5s/\<word\>/ where 5 is vi_arg1. (see also 49.)
45. Improved single line performance by roughly 3x. Syntax highlight will not
render anything beyond the terminal columns.
If there is a line in the file that has say 200K characters, the performance
will not degrade (except the order option set, which is by default). Please do
not underestimate the difficulty of such endeavor, nextvi is required to take
all these operations into account:
1. Bidi text direction.
2. Multibyte UTF-8. Double width chars.
3. Variable width tabulations that can change
throughout the line based on surroundings.
4. Reordering of characters. (regex rules)
5. Syntax highlight.
No other text editor has ever done all these requirements at once, therefore
come to appreciate what is provided here and the level of performance.
See also PERFORMANCE section at the end of the file.
Because syntax highlight is bounded to the terminal dimensions some patterns
may not be possible to match without rendering past the screen. To address
this problem you may need some extra patch, see PATCHES section below.
46. Added key to abandon insert discarding any changes keybind: ^x
That means there are 4 ways to exit insert|prompt now: ^x, ^c, ESC, kj
RATIONALE: unlike others, ^x will not leave file marked as modified.
47. Added Russian keymap, and changed how xkmap_alt works, now z + vi_arg1 in
normal mode will switch what keymap ^f key changes, so for example 1 = fa 2 =
ru. If you need some other language kmap just make one yourself. Look into
kmap.h it's really simple. kmap also has digraph support, that list has been
expanded to include non printing characters. Naming resembles ASCII caret
notation, so for example 03 is key ^c to be able to type 03 enter insert, use
^k to go into digraph mode, type then " c" (space+c) Essentially space is
representing typeable ctrl key. The reason this is useful is if you are
using  key to make macros there has to be a way to type these control
characters. Alternatively, it's possible to type the character literal using
^a + ^(insertkeyhere). In neatvi this was binded to ^v, currently ^v is used
for 48. Nextvi shall keep both, the digraph way and ^a for clarity and better
user reference, no more searching ascii keycodes online if kmap has it in the
clear.
48. Improvement to change 37. Now when in prompt/insert ^v will bring up the
latest command from history. Also the history works for searches via / or ?
the same way. keybind vv does the same but from normal mode, to save time.
Pressing ^v again goes to next string.
49. Added ability to get more than 1 word for keybinds ^a ^] ^p vr vt v/
specified by number (vi_arg1). Regex control chars will be escaped.
vr vt v/ ^] ^p will grab word(s) only if vi_arg1 >= 1 otherwise the keybind
will perform a default cursor independent action.
50. Added ability to edit the line while in insert mode such that backspace
can delete all the characters on the line, when no more characters left the
line will be wrapped onto the next one. This is behavior you can expect from
95% of editors, now nextvi is not an exception. The similar change was done for
^w keybind.
51. Added key to show the registers and their contents. keybind: R
If the length of a register is bigger than the terminal, specify vi_arg1 to
to shift the printing position by half the screen size vi_arg1 times.
52. Made feature of reverse text highlight toggleable via new ex option "hlr"
53. Added a key to disable autoindent. keybind: va
This is necessary sometimes if you want to paste from system clipboard.
54. Removed full names of ex commands and options, (seriously, who uses that?)
now only short and fast to type abbreviations work.
55. Substitute undo-redo point return to where command was issued initially
if the affected area is greater than terminal rows.
56. Modified regex engine to support more than 1 character in negated char
expressions. For example [^&&abc] where ^&& is the not+and. This will treat
"abc" as !(a && b && c) logically. Everything in brackets preceding && will
be processed according to standard negated char exp.
57. Added ^l key in insert to clean the terminal and start at line 0, remove
distractions and focus on typing. Useful when running ex via vi -e
58. Added v/ key in normal, which can grab the current word(s) under the
cursor into prompt and set the current search string. If valid, the input
will be used for all search related operations in vi. (see 49.)
59. Added ability to remember scroll amount for ^e and ^y keys (specified by
vi_arg1).
Advantage of ^e and ^y over using ^u and ^d is keeping the same vi_col position.
60. Removed bracket classes from regex. Not useful, hard to customize, buggy,
error prone mess. Doesn't add any new functionality to the regex engine that
can't be achieved without it.
61. Nextvi special character escapes work mostly the same way everywhere
except the following situations:
- Escapes in regex bracket expressions. This isn't posix but it solves
couple of issues that were bugged previously, like escaping | in ex
substitution command, properly counting number of groups in rset.
- A single slash requires 2 slashes, and so on.
- rset_make() requires for ( to be escaped if used inside [] brackets.
- In ex prompt the only separator is "|" character and there is some
counter intuitive handling of escapes done that has to be explained.
1: to use ex separator input "|"
2: to get "|" into the result string input = "\\|"
3: to get "\|" into the result string input = "\|"
4: to get "\\|" into the result string input = "\\\|"
5: to get "\\\|" into the result string input = "\\\\|"
A careful reader will spot that the rules above don't exactly
follow the well established rule - "2 slashes make one". This is
because to allow any combination of "|" and "" one would have to
force a "4 slashes make one" rule or implement special case parsing
for each ex command (examine ex_arg() in neatvi). The latter sucks, so
instead we make this special case for ex prompt that applies only to "|"
and "". In summary, just remember what clauses (2:) and (3:) state, for
cases where there are lots of slashes the rule is input minus 1 of the ""
equals the output string. Keep in mind that various ex functions, for example
regex will consume extra slashes internally if there is more than 1 after
another. If we wanted to match string "\\|" input will be: "\\\\\\|"
"\\\|" will get us "\\|" into regex, but the input needs 3 more slashes
because +1 rule (5:) +2 rule 2 slashes make 1. Timeline: "\\\\\\|" ->
rule(5:) "\\\\\|" -> rule(regex:2 = 1) "\\\|" = rule(regex:"\|" = "|")
"\\|" = this is the literal we wanted to match!
62. Added syntax highlighting continuation options. See the struct highlight
in vi.h. The ^ anchor in the regular expression has an important property of
being able to efficiently exclude some sub expression from being recomputed
during the continuation. Take advantage of it when you can.
63. New ex option "hlp", will highlight the closest pair of symbols {([
from the cursor, the same way % key works. This feature exists to demonstrate
complex syntax highlighting capabilities.
64. Ex options "hll", "hlw", "hlp" fully customizable in conf.c on per ft
basis the same way you customize per ft highlighting. They must have
highlight->func struct member set. If ft does not provide a spot in hl, the
latter feature will not work on that ft, regardless of ex options being set.
65. Added a key to quickly access :! prompt. keybind: v;
Removed "make" ex command. Commands like these are not wanted, nextvi shall
provide a more general purpose solution for the user, like the keybind v; for
example.
66. Added new ex option "grp". The following allows definition of target search
group for / ? n N (31.) autocomplete and ex substitution. This becomes
necessary when the result of regex search is to be based on some group rather
than default match group. For example you want to search for the whole line
but exclude the tabs at the beginning of the line, use regex like this:
[ ]+(.[^ ]+) since only the capture result for 2nd group matters use
the "grp" like this: :se grp=4 .The number 4 is important, it is calculated
using: grpnum * 2, to get the second group number do: 2 * 2 = 4. The first
group is always 2, 1 * 2 = 2, :se grp=2 gets you back to default group/behavior.
67. Undo and Redo commands (u,^r) may take optional vi_arg1 which repeats the
operation N times.
68. Search motions do not terminate with error if the count is greater than
number of instances found. Last possible match will be used. Important when
you don't know exactly how many matches there are, does not mean there aren't
any at all, greedy behavior opens up new use cases.
69. New ex command "tp", when arg given immediately executes the macro defined
by arg. It can run any vi normal command and execute insert statements.
The advantage of tp over traditional macros is in the ability to bypass the
macro queue and run independently. In a way, macro executed by tp exercises
the same causality as running C code directly.
70. Added key to list through the buffers. keybind: ^n
vi_arg1 changes the direction of ^n.
71. Added keybinds ^\ and ^] in insert mode to allow up and down scrolling
and quick regex searches without exiting insert. Use ^z to set the mark
where the regex begins. keybind ^o allows access to a secondary ex prompt or
new vi instance so that it becomes possible to switch to vi from ex or postpone
some operation, do something else and come back later.
This feature is implemented using functionality provided by 69. and exists to
prove and demonstrate that vi is to some extent REENTRANT as in single
threaded recursion case, where operations can be safely executed in a
separate instance. Still, a great amount of care and understanding is needed
when making such complicated macros.
LESSER KNOWN FEATURES
---------------------
- "Ever tried reading the source code?"
Yes, that is a lesser known feature, what did you expect?
Jokes aside (with a level of truth to it), these features exist in many other
vi implementations but neither man pages cover their functionality in an
understandable language, describe it here instead.
- @ macros:
1. Type out the macro or load from file such that it is in some vi buffer.
2. Use keybind "ayy on the macro, this will store it in register 'a'
3. Use a to play it back, where a stands for that 'a' register
4. @ repeats the last macro on next line
- substitution backreference:
This inserts the text of matched group specified by \x where x is
group number. Example:
this is an example text for subs and has int or void
:%s/(int)\\|(void)/pre\0after
this is an example text for subs and has preintafter or void
:%s/(int)\\|(void)/pre\2after/g
this is an example text for subs and has prepreafterafter or prevoidafter
- ex global command:
same syntax as ex substitution command, but instead of replacement
string it takes an ex command after the / / enclosed regex.
Example: remove empty lines
:g/^$/d
Try doing similar with substitution command - will not work as removing '\n'
without deleting the line is invalid, but it will work with global command.
- search motions:
? and / searches have the ability to be used as motions. This seems very
counter intuitive and one would have never ever figure out that this
feature even exists, unless noted. Even if you read the source code it's
very easy to miss. How to use: optionally specify vi_arg1, specify the motion
using it's keybind, then do / or ? and type out the search term.
The motion ends on the first match by default (no vi_arg1 specified).
The optional vi_arg1 determines how many matches of the term to skip until the
motion ends. Example: you see that the next 10 lines have the word "int"
which is included 3 times. You want to delete text until the 3rd
instance of "int" keybind would be 3d/int . Likewise you can opt out
of the "specify motion" part and just use / or ? with vi_arg1 to perform
specific searches.
- Majestic EXINIT environment variable
At the zenith of your vi/nextvi education you'll find that EXINIT can be
used to achieve arbitrary level of customization. Using new ex command "tp"
any sequence of vi/ex commands can be performed at startup. This is where
real "groking vi" starts.
Example 1:
There is a dictionary file (assume vi.c), which we always want to have indexed
at startup for autocomplete feature in 31.
export EXINIT="e ./vi.c|tp i|u|bx 1|bx"
The last "bx" commands delete the vi.c buffer. "u" can be used to unmark vi.c
as being modified. To keep it around as a buffer remove the "bx" commands.
Example 2:
Load your shell's history into vi's history buffer and adjust the data such
that it is usable by appending ! at the beginning of command and escaping the
"|"
pipes the way ex prompt expects them (see 61.)
export EXINIT="e /root/.ash_history|tp yG:p:%s/^/!\\\|%s/ \\| / \\\\\\\| /g
qq|bx 1|bx|ft"
Congratulations, vi has unofficially replaced your shell's frontend.
Example 3:
Setup some custom @ macros in your favorite registers.
export EXINIT="e|tp io{
}kA|tp 1G|tp 2"ayy"
This macro gets loaded into register a, when a is executed the macro will
create { and closing } below the cursor leaving cursor in insert mode in
between the braces. This is something you would commonly do in C like
programming language.
- CRITICAL - the new line inside the EXINIT string is literal, it's best to
store the export command in a .sh file and set it using:
$ . ./init.sh
Otherwise this examples 2 and 3 may not work!!!
To suppress EXINIT variable use -s commandline flag.
- Uppercase registers
In vi uppercase registers append to the lowercase register instead of
overwriting the register completely. This is very useful, for example,
use global and yank ex commands together:
:g/searchterm/ya A
Now we can use "ap and paste all the lines matched by the regex. Depending
on the usecase simply :g/searchterm/ya or :g/searchterm/p may also be enough.
PATCHES
-------
New functionality can be obtained through optional patches provided in the
patches branch. If you have a meaningful contribution and would love to be
made public the patch can be submitted via email or github pull request.
FAQ:
----
Q1: What's the best way to learn vi/nextvi?
A1: First ensure you know basic movements hjkl this would suffice.
Start reading vi.c don't worry about the rest of this readme until later.
Running ./vi vi.c use / and n N keybinds for search and look for switch cases.
The keybinds are encoded to be intuitive. Once you find some case that looks
like a keybind read the code, if you don't understand try to reproduce the
keybind to better understand what the code does. You have to do this, if you
omit this step you will never be able to realize the full potential that the
software provides. It's not desirable to live in the dark using this
software for the next 10 years only to find that for example, ^p in insert
mode exists and is very useful. If you can't figure how to use the keybind at
least you would know at the back of your mind that there is something there,
realization will come later. It's better to skim look through the switch
cases than to never even open it. This isn't an excuse, but a deliberate
design goal, where the user reads the code in order to achieve the full
control he/she desires.
LOC:
+--------------+---------------------+
| 569 kmap.h | keymap translation |
| 425 vi.h | definitions/aux |
+--------------+---------------------+
| 577 uc.c | UTF-8 support |
| 336 term.c | low level IO |
| 298 conf.c | hl/ft/td config |
| 637 regex.c | extended RE |
| 590 lbuf.c | file/line buffer |
| 1125 ex.c | ex options/commands |
| 2080 vi.c | normal mode/general |
| 725 led.c | insert mode/output |
| 386 ren.c | positioning/syntax |
| 6754 total | wc -l *.c |
+--------------+---------------------+
The code is devised to be unquestionable.
You will be able to read, understand and modify this code faster.
Come back to this readme regularly as it documents more advanced behavior.
Q2: What does it mean when I call feature X a macro?
A2: It's the kind of shortcut that does not change the core functionality,
but rather reuses the core functionality. Usually macro features are
implemented in 1 or a few lines of code. Notably, they tend to use function
term_push(), but it's not required. Because they are macros they may run
suboptimally or not handle every possible edge case. When calling a macro
feature from another macro, the results are pushed back, which means the
macro feature will always execute last, see (69.).
These features are considered a macro: 5. 6. 16. 17. 18. 19. 20.
21. 26. 27. 34. 46. 50. 65. 71.
Q3: Keybind with CTRL does not work?
A3: vi is reading ASCII codes sent by the terminal. Depending on the
keyboard, the ASCII code could be another key combination. It was reported
that "^^" (Ctrl + ^) can be achieved on some system with "^6". If something
doesn't work, have a look at the layout of an american/british keyboard and
try to reproduce the keybind as if you have an american/british keyboard.
Q4: Why nextvi instead of vim?
A4: I prefer customization in source code, Vim is considered harmful.
Q5: Why not distribute as patches, like on suckless.org?
A5: It's hard to maintain. Simply put, there are too many changes
to keep track of if compared to original neatvi.
Q6: Why are keybinds encoded as pure switch cases instead of more
suckless.org style keybind function table dispatch?
A6: Because we want small efficient code that is easy to write.
In nextvi many keybinds interoperate so that they can do multiple tasks at
various conditions. Use of goto is encouraged, it is simply impossible to
achieve this behavior in a sensible way otherwise. Suckless code style
philosophy crumples when requirements are as complex as what vi needs to be
able to do. In other words, it depends - but if the standard of C provides
means to implementing things cleaner and faster you should use the smallest
form factor possible. In retrospect, it may be harder to find the
implementation itself, but once you do there is nothing else hidden from you.
The result is unabstracted program control flow that can be easily read and
modified reducing the risks of unforeseen side effects.
Q7: General philosophy?
A7: User is programmer, hacker culture.
In most text editors, flexibility is a minor or irrelevant design goal.
Nextvi is designed to be flexible where the editor adapts to the user needs.
This flexibility is achieved by heavily chaining basic commands and allowing
them to create new ones with completely different functionality. Command
reuse keeps the editor small without infringing on your freedom to quickly
get a good grasp on the code. If you want to customize anything, you should
be able to do it using the only core commands or a mix with some specific C
code for more difficult tasks. Simple and flexible design allows for straight
forward solutions to any problem long term and filters bad inconsistent ideas.
Q8: Something, something - pikevm
A8: Pikevm is a complete rewrite of nextvi's regex engine for the purposes of
getting rid of backtracking and severe performance and memory constraints.
Pikevm guarantees that all regular expressions are computed in constant space
and O(n+k) time where n is size of the string and k is some constant for the
complexity of the regex ie. number of state transitions. It is important to
understand that it does not mean that we run at O(n) linear speed, but rather
the amount of processing time & memory usage is distributed evenly and linearly
throughout the string, the k constant plays a big role. If you are familiar
with radix sort algorithms this follows the same idea.
Q: What are the other benefits?
A: For example, now it is possible to compute a C comment /* n */ where n can
be an infinite number of characters. Of course this extends to every other
valid regular expression.
Q: New features pikevm supports?
A: Additionally, pikevm supports PCRE style non capture group (?:) and lazy
quantifiers like .*? and .+?? because they were easy to implement and allow
for further regex profiling/optimization.
Q: NFA vs DFA (identify)
A: pikevm = NFA backtrack = DFA
Q: What's wrong with original implementation?
A: Nothing except it being slow and limited. My improved version of Ali's DFA
implementation ran 3.5X faster in any case, however I found a bug with it
where quested "?" nested groups compute wrong submatch results. To fix this
problem, it would require to undo a lot of optimization work already done,
basically going back to how slow Ali's implementation would be. The reason
this was spotted so late was because this kind of regex wasn't used before,
so I never tested it. Other than that I think submatch extraction is correct
on other cases. Pikevm does not have this bug, so it will be used as main
regex engine from now on, unless dfa ever finds a proper fix. Honestly, this
change isn't so surprising, as I was working on pikevm a few months prior, to
favor a superior algorithm.
You can still find that code here (likely with no updates):
https://github.com/kyx0r/nextvi/tree/dfa_dead
As a downside, NFA simulation looses the DFA property of being able to
quickly short circuit a match, as everything runs linearly and at constant
speed, incurring match time overhead. Well optimized DFA engine can
outperform pikevm, but that is rather rare as they got problems of their own.
For example as independently benchmarked, dfa_dead runs only 13% faster than
pikevm and that is stretching the limit of what is physically possible on a
table based matcher. Can't cheat mother nature, and if you dare to try she's
unforgiving at best.
Supplementary reading by Russ Cox:
https://swtch.com/~rsc/regexp/regexp1.html
PERFORMANCE
-----------
Stress test:
1. Compile both versions with gcc -O2 -g (no customizations done)
2. Open Nextvi's vi.c and hold ^d until the end of the file
3. Capture the results with cachegrind
4. To find out what these values mean see:
https://valgrind.org/docs/manual/cg-manual.html
NEATVI:
--------------------------------------------------------------------------------
| I refs: 1,030,137,070
| I1 misses: 122,660
| LLi misses: 1,298
| I1 miss rate: 0.01%
| LLi miss rate: 0.00%
|
| D refs: 672,331,702 (373,790,802 rd + 298,540,900 wr)
| D1 misses: 473,511 ( 215,088 rd + 258,423 wr)
| LLd misses: 7,836 ( 1,116 rd + 6,720 wr)
| D1 miss rate: 0.1% ( 0.1% + 0.1% )
| LLd miss rate: 0.0% ( 0.0% + 0.0% )
|
| LL refs: 596,171 ( 337,748 rd + 258,423 wr)
| LL misses: 9,134 ( 2,414 rd + 6,720 wr)
| LL miss rate: 0.0% ( 0.0% + 0.0% )
--------------------------------------------------------------------------------
NEXTVI:
--------------------------------------------------------------------------------
| I refs: 364,197,180
| I1 misses: 93,186
| LLi misses: 1,228
| I1 miss rate: 0.03%
| LLi miss rate: 0.00%
|
| D refs: 142,125,211 (100,909,090 rd + 41,216,121 wr)
| D1 misses: 140,991 ( 81,656 rd + 59,335 wr)
| LLd misses: 6,043 ( 1,855 rd + 4,188 wr)
| D1 miss rate: 0.1% ( 0.1% + 0.1% )
| LLd miss rate: 0.0% ( 0.0% + 0.0% )
|
| LL refs: 234,177 ( 174,842 rd + 59,335 wr)
| LL misses: 7,271 ( 3,083 rd + 4,188 wr)
| LL miss rate: 0.0% ( 0.0% + 0.0% )
--------------------------------------------------------------------------------
Further optimization:
1. To create the most optimal exe, enable PGO optimizations by compiling via
./build.sh pgobuild which can lead to a significant performance boost on some
application specific tasks. Feel free to adjust build.sh and the sample data
on which it's being trained on, though default probably already good enough.
2. To improve nextvi's performance, shaping, character reordering, and
syntax highlighting can be disabled by defining the EXINIT environment
variable as "se noshape | se noorder | se nohl | se td=+2".
Favorite quotes:
--------------------------------------------------------------------------------
"All software sucks, but some do more than others."
- Kyryl Melekhin
"Educated decisions assert the quantitative quality first."
- Kyryl Melekhin
"It's possible that I understand better what's going on, or
it's equally possible that I just think I do."
- Russ Cox
"Vigorous writing is concise. A sentence should contain no unnecessary words
[and] a paragraph no unnecessary sentences, for the same reason that a drawing
should have no unnecessary lines and a machine no unnecessary parts. This
requires not that the writer make all his sentences short, or that he avoid
all detail and treat his subjects only in outline, but that every word tell."
- Elements of Style, William Strunk, Jr. - 1918
--------------------------------------------------------------------------------
CREDITS
=============
Programming
-------------
Kyryl Melekhin (kyx0r)
Ali Gholami Rudi (aligrudi)
Documentation / Design / Testing
-------------
Kyryl Melekhin (kyx0r)
Proofreading
-------------
Kyryl Melekhin (kyx0r)
Cédric (Vouivre)
Special Thanks
-------------
Ali Gholami Rudi (vi https://github.com/aligrudi/neatvi)
ArmaanB (bsd test)
aabacchus (build.sh)
illiliti (build.sh)
git-bruh (feedback)
and all users, posters & haters :/
|
