flua: move lposix back into flua for now

The real luaposix is structured differently, and our builtin version
tends to model that in its own special way. Build it back in until we
get the shlib mod

flua: move lposix back into flua for now

The real luaposix is structured differently, and our builtin version
tends to model that in its own special way. Build it back in until we
get the shlib model just right to unbreak nuageinit, among other things.

This is a partial revert of the below-referenced commit; the other two
modules are simple, though, and were accurately split out.

Reported by: markj
Fixes: b11a5709ec2b6 ("flua: kick out the remaining builtin modules")

show more ...

flua: kick out the remaining builtin modules

Bootstrap flua has some magic now to handle modules by building them in
and discovering them via linker sets. This is slightly cleaner than
always build

flua: kick out the remaining builtin modules

Bootstrap flua has some magic now to handle modules by building them in
and discovering them via linker sets. This is slightly cleaner than
always building them in and baking them into loadedlibs for both
bootstrap and system flua.

Adjust the stand build now that these three libs have their own new
homes.

Reviewed by: bapt, emaste
Differential Revision: https://reviews.freebsd.org/D51891

show more ...

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
canno

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
cannot dlsym(), so we can't really discover the names of our newly
builtin modules. Instead, just build out a linker set with all of our
luaopen_*() functions to register everything up-front.

Building in all of the modules isn't strictly necessary, but it means
that we have an example of how to add a bootstrap module everywhere you
go and one doesn't need to consider whether bootstrap flua can use a
module when writing scripts. On my build machine, the consequence on
our binary size is an increase from around 1.6M -> 1.9M, which isn't
really that bad.

.lua modules can install into their usual path below $WORLDTMP/legacy
and we'll pick them up automagically by way of the ctor that sets up
LUA_PATH early on.

This re-lands bootstrap module support with a more sensible subset, and
after having verified that it cross-builds fine on macOS and Linux -- we
cannot do libfreebsd on !FreeBSD because it's more system header
dependant. We also need to bootstrap libmd to bring in libhash, and
libucl + libyaml.

Reviewed by: bapt, emaste (both previous version)
Differential Revision: https://reviews.freebsd.org/D51890

show more ...

Revert "flua: support our flua modules in the bootstrap flua"

This reverts commit 1953a12ee2cde1afacb3e3f7612d89695c96e04f, because it
cannot work at all on macOS without more work, at a minimum. W

Revert "flua: support our flua modules in the bootstrap flua"

This reverts commit 1953a12ee2cde1afacb3e3f7612d89695c96e04f, because it
cannot work at all on macOS without more work, at a minimum. We use
linker sets for module discovery, but we don't have a version of this
that works for mach-o at the moment.

show more ...

Revert "flua: kick out the remaining builtin modules"

This reverts commit 80ada959004c4386880e47b11618f8abfc2d80e1, because
bootstrap flua is about to get backed out.

flua: kick out the remaining builtin modules

Bootstrap flua has some magic now to handle modules by building them in
and discovering them via linker sets. This is slightly cleaner than
always build

flua: kick out the remaining builtin modules

Bootstrap flua has some magic now to handle modules by building them in
and discovering them via linker sets. This is slightly cleaner than
always building them in and baking them into loadedlibs for both
bootstrap and system flua.

Adjust the stand build now that these three libs have their own new
homes.

Reviewed by: bapt, emaste
Differential Revision: https://reviews.freebsd.org/D51891

show more ...

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
canno

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
cannot dlsym(), so we can't really discover the names of our newly
builtin modules. Instead, just build out a linker set with all of our
luaopen_*() functions to register everything up-front.

Building in all of the modules isn't strictly necessary, but it means
that we have an example of how to add a bootstrap module everywhere you
go and one doesn't need to consider whether bootstrap flua can use a
module when writing scripts. On my build machine, the consequence on
our binary size is an increase from around 1.6M -> 1.9M, which isn't
really that bad.

.lua modules can install into their usual path below $WORLDTMP/legacy
and we'll pick them up automagically by way of the ctor that sets up
LUA_PATH early on.

Reviewed by: bapt, emaste
Differential Revision: https://reviews.freebsd.org/D51890

show more ...

lposix: Clean up the posix namespace definitions

The posix module is subdivided according to C headers; for instance,
posix.unistd contains routines available from unistd.h, such as
chown(2).

A qui

lposix: Clean up the posix namespace definitions

The posix module is subdivided according to C headers; for instance,
posix.unistd contains routines available from unistd.h, such as
chown(2).

A quirk of our implementation is that each of the modules is a direct
entry in the global table. That is, there is no "posix" table.
Instead, "posix.foo" and "posix.bar.baz" are both top-level tables.
This is surprising and goes against Lua's shorthand of using "." to
access keys in a table. lua-posix also doesn't work this way.

Rework things so that "posix" and "posix.sys" are proper tables.
Existing flua code which uses require() to bind posix submodules to a
name will be unaffected. Code which accesses them directly using
something like _G["posix.sys.utsname"].uname() will be broken, but I
don't think anything like that exists. In particular, it is now
possible to call posix.sys.utsname.uname() without any require
statements.

Reviewed by: imp, bapt
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D51158

show more ...

flua: lposix: add fnmatch function

The fnmatch function matches a string against a shell-style filename
pattern. It is a complex function and cannot easily be implenented
using regular expressions.

flua: lposix: add fnmatch function

The fnmatch function matches a string against a shell-style filename
pattern. It is a complex function and cannot easily be implenented
using regular expressions. Adding fnmatch to flua increases the amd64
binary by less than 1 KB.

Approved by: markj
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D46849

show more ...

flua: lposix: add more useful functions for general purpose scripts

unistd:
- _exit
- close
- fork
- getpid
- pipe
- read
- write

libgen:
- basename, dirname

stdlib:
- realpath

These are sufficie

flua: lposix: add more useful functions for general purpose scripts

unistd:
- _exit
- close
- fork
- getpid
- pipe
- read
- write

libgen:
- basename, dirname

stdlib:
- realpath

These are sufficient for a number of real world scenarios. In our first
application of them, we use the libgen+stdlib additions to grab the
script dir based on argv[0]. The unistd assortment is then used to
outsource a bunch of work to forks and report back to the main process.

Reviewed by: emaste, imp
Differential Revision: https://reviews.freebsd.org/D39083

show more ...

flua: make the ucl module a dynamic module

Reviewed by: markj
Differential Revision: https://reviews.freebsd.org/D46569

flua: Add wrappers for sys/utsname.h

This allows one to invoke uname from lua scripts.

Reviewed by: bapt, kevans, emaste
MFC after: 1 month
Differential Revision: https://reviews.freebsd.org/D42017

flua: add fbsd module

This module is bundled into flua, it only provides for now the exec
function. The point of the function is to be able to execute a program
without actually executing a shell.

flua: add fbsd module

This module is bundled into flua, it only provides for now the exec
function. The point of the function is to be able to execute a program
without actually executing a shell.

to use it:
fbsd.exec({"id", "bapt"})

Reviewed by: manu
Differential Revision: https://reviews.freebsd.org/D41840

show more ...

Remove $FreeBSD$: one-line .c comment pattern

Remove /^/[*/]\s*\$FreeBSD\$.*\n/

flua: add ucl library

libucl comes with a Lua library binding. Build it into flua.

This lets us parse/generate config files in the various formats supported by
libucl with flua. For example, the

flua: add ucl library

libucl comes with a Lua library binding. Build it into flua.

This lets us parse/generate config files in the various formats supported by
libucl with flua. For example, the following script will detect the format of
an object written to stdin as one of UCL config, JSON, or YAML and write it to
stdout as pretty-printed JSON:

local ucl = require('ucl')
local parser = ucl.parser()
parser:parse_string(io.read('*a'))
local obj = parser:get_object()
print(ucl.to_format(obj, 'json'))

Reviewed by: kevans, pstef
Approved by: mmacy (mentor)
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D25009

show more ...

flua: implement chmod

Lua does not provide a native way to change the permission of a file.

Submitted by: Yang Wang <2333@outlook.jp>
Reviewed by: kevans
Sponsored by: The FreeBSD Foundation
Differ

flua: implement chmod

Lua does not provide a native way to change the permission of a file.

Submitted by: Yang Wang <2333@outlook.jp>
Reviewed by: kevans
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D24036

show more ...

Add flua to the base system, install to /usr/libexec

FreeBSDlua ("flua") is a FreeBSD-private lua, flavored with whatever
extensions we need for base system operations. We currently support a subset

Add flua to the base system, install to /usr/libexec

FreeBSDlua ("flua") is a FreeBSD-private lua, flavored with whatever
extensions we need for base system operations. We currently support a subset
of lfs and lposix that are used in the rewrite of makesyscall.sh into lua,
added in r354786.

flua is intentionally written such that one can install standard lua and
some set of lua modules from ports and achieve the same effect.

linit_flua is a copy of linit.c from contrib/lua with lfs and lposix added
in. This is similar to what we do in stand/. linit.c has been renamed to
make it clear that this has flua-specific bits.

luaconf has been slightly obfuscated to make extensions more difficult. Part
of the problem is that flua is already hard enough to use as a bootstrap
tool because it's not in PATH- attempting to do extension loading would
require a special bootstrap version of flua with paths changed to protect
the innocent.

src.lua.mk has been added to make it easy for in-tree stuff to find flua,
whether it's bootstrap-flua or relying on PATH frobbing by Makefile.inc1.

Reviewed by: brooks, emaste (both earlier version), imp
Differential Revision: https://reviews.freebsd.org/D21893

show more ...

flua: move lposix back into flua for now

The real luaposix is structured differently, and our builtin version
tends to model that in its own special way. Build it back in until we
get the shlib mod

flua: move lposix back into flua for now

The real luaposix is structured differently, and our builtin version
tends to model that in its own special way. Build it back in until we
get the shlib model just right to unbreak nuageinit, among other things.

This is a partial revert of the below-referenced commit; the other two
modules are simple, though, and were accurately split out.

Reported by: markj
Fixes: b11a5709ec2b6 ("flua: kick out the remaining builtin modules")

show more ...

flua: kick out the remaining builtin modules

Bootstrap flua has some magic now to handle modules by building them in
and discovering them via linker sets. This is slightly cleaner than
always build

flua: kick out the remaining builtin modules

Bootstrap flua has some magic now to handle modules by building them in
and discovering them via linker sets. This is slightly cleaner than
always building them in and baking them into loadedlibs for both
bootstrap and system flua.

Adjust the stand build now that these three libs have their own new
homes.

Reviewed by: bapt, emaste
Differential Revision: https://reviews.freebsd.org/D51891

show more ...

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
canno

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
cannot dlsym(), so we can't really discover the names of our newly
builtin modules. Instead, just build out a linker set with all of our
luaopen_*() functions to register everything up-front.

Building in all of the modules isn't strictly necessary, but it means
that we have an example of how to add a bootstrap module everywhere you
go and one doesn't need to consider whether bootstrap flua can use a
module when writing scripts. On my build machine, the consequence on
our binary size is an increase from around 1.6M -> 1.9M, which isn't
really that bad.

.lua modules can install into their usual path below $WORLDTMP/legacy
and we'll pick them up automagically by way of the ctor that sets up
LUA_PATH early on.

This re-lands bootstrap module support with a more sensible subset, and
after having verified that it cross-builds fine on macOS and Linux -- we
cannot do libfreebsd on !FreeBSD because it's more system header
dependant. We also need to bootstrap libmd to bring in libhash, and
libucl + libyaml.

Reviewed by: bapt, emaste (both previous version)
Differential Revision: https://reviews.freebsd.org/D51890

show more ...

Revert "flua: support our flua modules in the bootstrap flua"

This reverts commit 1953a12ee2cde1afacb3e3f7612d89695c96e04f, because it
cannot work at all on macOS without more work, at a minimum. W

Revert "flua: support our flua modules in the bootstrap flua"

This reverts commit 1953a12ee2cde1afacb3e3f7612d89695c96e04f, because it
cannot work at all on macOS without more work, at a minimum. We use
linker sets for module discovery, but we don't have a version of this
that works for mach-o at the moment.

show more ...

Revert "flua: kick out the remaining builtin modules"

This reverts commit 80ada959004c4386880e47b11618f8abfc2d80e1, because
bootstrap flua is about to get backed out.

flua: kick out the remaining builtin modules

Bootstrap flua has some magic now to handle modules by building them in
and discovering them via linker sets. This is slightly cleaner than
always build

flua: kick out the remaining builtin modules

Bootstrap flua has some magic now to handle modules by building them in
and discovering them via linker sets. This is slightly cleaner than
always building them in and baking them into loadedlibs for both
bootstrap and system flua.

Adjust the stand build now that these three libs have their own new
homes.

Reviewed by: bapt, emaste
Differential Revision: https://reviews.freebsd.org/D51891

show more ...

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
canno

flua: support our flua modules in the bootstrap flua

This version builds every module into the flua binary itself, since all
of the bootstrap tools are built -DNO_SHARED. As a result, we also
cannot dlsym(), so we can't really discover the names of our newly
builtin modules. Instead, just build out a linker set with all of our
luaopen_*() functions to register everything up-front.

Building in all of the modules isn't strictly necessary, but it means
that we have an example of how to add a bootstrap module everywhere you
go and one doesn't need to consider whether bootstrap flua can use a
module when writing scripts. On my build machine, the consequence on
our binary size is an increase from around 1.6M -> 1.9M, which isn't
really that bad.

.lua modules can install into their usual path below $WORLDTMP/legacy
and we'll pick them up automagically by way of the ctor that sets up
LUA_PATH early on.

Reviewed by: bapt, emaste
Differential Revision: https://reviews.freebsd.org/D51890

show more ...

lposix: Clean up the posix namespace definitions

The posix module is subdivided according to C headers; for instance,
posix.unistd contains routines available from unistd.h, such as
chown(2).

A qui

lposix: Clean up the posix namespace definitions

The posix module is subdivided according to C headers; for instance,
posix.unistd contains routines available from unistd.h, such as
chown(2).

A quirk of our implementation is that each of the modules is a direct
entry in the global table. That is, there is no "posix" table.
Instead, "posix.foo" and "posix.bar.baz" are both top-level tables.
This is surprising and goes against Lua's shorthand of using "." to
access keys in a table. lua-posix also doesn't work this way.

Rework things so that "posix" and "posix.sys" are proper tables.
Existing flua code which uses require() to bind posix submodules to a
name will be unaffected. Code which accesses them directly using
something like _G["posix.sys.utsname"].uname() will be broken, but I
don't think anything like that exists. In particular, it is now
possible to call posix.sys.utsname.uname() without any require
statements.

Reviewed by: imp, bapt
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D51158

show more ...