jn
/
punkshell
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
Browse Source

punk_main.tcl: add 'src' package_mode for loading unbuilt source modules and libraries 

The new 'src' package_mode loads unbuilt modules and libraries from the project's src/ tree (src/modules, src/lib, src/bootsupport, src/vendormodules) instead of built output. It differs from 'dev' mode which points at built <projectroot>/modules. The src mode also runs an inline #modpod package-ifneeded scanner using Tcl builtins only (since no punk modules are loaded at boot time) and sets package prefer latest so 999999.0a1.0 dev modules are preferred over stable bootsupport/vendored copies. The exhaustive permutation switch for package_mode parsing was replaced with token-by-token validation that scales to any number of modes. Verified: punk902z src launches an interactive shell with package provide punk returning 999999.0a1.0.
master
Julian Noble 1 week ago
parent
91178a458f
commit
3d2d098da9
2 changed files with 209 additions and 25 deletions
Whitespace
Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL
Split View
Diff Options
Show Stats Download Patch File Download Diff File
  1. 3
      src/vfs/AGENTS.md
  2. 231
      src/vfs/_config/punk_main.tcl

3
src/vfs/AGENTS.md
Unescape View File

@ -15,6 +15,9 @@ VFS (Virtual File System) folders define the runtime payloads that get wrapped i
- `*.vfs` folders are build artifacts consumed by `tclsh src/make.tcl project`.
- VFS content must be compatible with the target platform runtime.
- `_vfscommon.vfs/` is an auto-generated merge of common libraries; manually editing it will be overwritten.
- `_config/punk_main.tcl` accepts an optional package_mode argument before the subcommand. Valid modes are `dev`, `os`, `internal`, and `src`, dash-delimited in priority order (e.g `src-internal`). `internal` is always appended if omitted.
- `src` mode loads unbuilt modules and libraries from the project's `src/` tree (`src/modules`, `src/lib`, `src/bootsupport`, `src/vendormodules`) instead of built output. It also runs an inline `#modpod` package-ifneeded scanner (using Tcl builtins only, since no punk modules are loaded at boot time) and sets `package prefer latest` so `999999.0a1.0` dev modules are preferred over stable bootsupport/vendored copies.
- `dev` mode loads built output from `<projectroot>/modules` and `<projectroot>/lib`; `src` mode loads unbuilt source from `<projectroot>/src/modules` etc. Use `src-dev-internal` to combine both with src priority.
## Work Guidance

231
src/vfs/_config/punk_main.tcl
Unescape View File

@ -273,8 +273,10 @@ apply { args {
set tm_additions_internal [list]
set tm_additions_dev [list]
set tm_additions_src [list]
set auto_path_additions_internal [list]
set auto_path_additions_dev [list]
set auto_path_additions_src [list]
set lc_auto_path [string tolower $::auto_path]
@ -357,47 +359,50 @@ apply { args {
# dev - refers to module and library paths relative to the project (executable path)
# os - refers to modules and library paths gleaned from ::env (TCLLIBPATH and TCL<MAJOR>_<MINOR>_TM_PATH)
# internal - refers to modules and libraries supplied from the mounted filesystem of a kit or zipfs based executable
# src - refers to unbuilt modules and libraries under the project's src/ tree (src/modules, src/lib, src/bootsupport, src/vendormodules)
# -----------------------------------------------------------------------------------------------------------
# Note that unlike standard 'package unknown' punk::libunknown does not stop searching for packages when a .tm file is found that matches requirements,
# The auto_path is still examined. (avoids quirks where higher versioned pkgIndex based package not always found)
# -----------------------------------------------------------------------------------------------------------
set all_package_modes [list dev os internal]
set all_package_modes [list dev os internal src]
#package_mode is specified as a dash-delimited ordered value e.g dev-os
#"internal" is the default and if not present is always added to the list
#i.e "dev-os" is equivalent to "dev-os-internal"
#"os" is equivalent to "os-internal"
#"internal-os" and "internal" are left as is.
#The effective package_mode has 1 2 or 3 members.
#The effective package_mode has 1 2 3 or 4 members.
# The only case where it has 1 member is if just "internal" is specified.
#This gives the number of permutations as how many ways to choose 3 items plus how many ways to choose 2 of the 3 items (one must be 'internal') plus the sole allowable way to choose 1
#for a total of 11 possible final orderings.
#(16 possible values for package_mode argument when you include the short-forms "",os,dev,os-dev,dev-os which always have 'internal' appended)
set test_package_mode [lindex $args 0]
#puts stderr "main.tcl test_package_mode: '$test_package_mode'"
switch -exact -- $test_package_mode {
internal -
os-internal - dev-internal - internal-os - internal-dev -
os-dev-internal - os-internal-dev - dev-os-internal - dev-internal-os - internal-os-dev - internal-dev-os {
#fully specified ('internal' is present)
set package_modes [split $test_package_mode -]
set arglist [lrange $args 1 end]
#Token-by-token validation: instead of exhaustively listing all permutations,
#split the first arg on dash and validate each token against the known mode set.
#This scales to any number of modes without enumerating every permutation.
set package_modes ""
set arglist ""
if {$test_package_mode eq ""} {
#empty first arg consumed as equivalent of 'internal'
set package_modes internal
set arglist [lrange $args 1 end]
} else {
set tokens [split $test_package_mode -]
set valid 1
foreach t $tokens {
if {$t ni $all_package_modes} {
set valid 0
break
}
}
os - dev - os-dev - dev-os {
#partially specified - 'internal' ommitted but implied at tail
set package_modes [list {*}[split $test_package_mode -] internal]
if {$valid && [llength $tokens] >= 1} {
set package_modes $tokens
if {"internal" ni $package_modes} {
lappend package_modes internal
}
set arglist [lrange $args 1 end]
}
default {
#empty first arg - or some unrelated arg
} else {
#not a package_mode - treat as subcommand
set package_modes internal
if {$test_package_mode eq ""} {
#consume the empty first arg as an equivalent of 'internal'
#don't consume any first arg that isn't recognised as a package_mode
set arglist [lrange $args 1 end]
} else {
set arglist $args
}
set arglist $args
}
}
#assert: arglist has had any first arg that is a package_mode (including empty string) stripped.
@ -532,6 +537,54 @@ apply { args {
}
}
#src mode: discover the project's src/ tree and add unbuilt module paths.
#Unlike dev mode (which points at built output in <projectroot>/modules),
#src mode points at the unbuilt source in <projectroot>/src/modules etc.
if {"src" in $package_modes} {
#reuse the dev-mode project root discovery (exe in bin/ -> backtrack 1)
set src_project_root ""
set normexe_dir_for_src [file dirname $normexe]
if {[file tail $normexe_dir_for_src] eq "bin"} {
set src_project_root [file dirname $normexe_dir_for_src]
} else {
set src_project_root $normexe_dir_for_src
}
#also check symlink target
if {$src_project_root eq ""} {
set nameexe_dir_for_src [file dirname [file normalize [info nameofexecutable]]]
if {[file tail $nameexe_dir_for_src] eq "bin"} {
set src_project_root [file dirname $nameexe_dir_for_src]
} else {
set src_project_root $nameexe_dir_for_src
}
}
if {$src_project_root ne "" && [file isdirectory [file join $src_project_root src]]} {
foreach p [list modules modules_tcl$tclmajorv] {
set modpath [file join $src_project_root src $p]
if {[file isdirectory $modpath] && $modpath ni $tm_additions_src} {
lappend tm_additions_src $modpath
}
}
#bootsupport modules (for boot-critical packages like punkcheck, punk::mix etc)
foreach p [list modules modules_tcl$tclmajorv] {
set modpath [file join $src_project_root src bootsupport $p]
if {[file isdirectory $modpath] && $modpath ni $tm_additions_src} {
lappend tm_additions_src $modpath
}
}
#vendormodules (vendored dependencies like voo)
foreach p [list vendormodules vendormodules_tcl$tclmajorv] {
set modpath [file join $src_project_root src $p]
if {[file isdirectory $modpath] && $modpath ni $tm_additions_src} {
lappend tm_additions_src $modpath
}
}
} else {
puts stderr "Warning - src mode: no src/ directory found relative to executable ($normexe_dir_for_src)"
}
}
@ -595,6 +648,13 @@ apply { args {
}
}
}
src {
foreach n $tm_additions_src {
if {$n ni $new_tm_path} {
lappend new_tm_path $n
}
}
}
os {
foreach n $external_tm_dirs {
if {$n ni $new_tm_path} {
@ -698,6 +758,42 @@ apply { args {
}
}
#src mode: add unbuilt source library paths from the project's src/ tree.
if {"src" in $package_modes && [info exists src_project_root] && $src_project_root ne ""} {
set platform [::punkboot::platform_generic]
set src_lib_base [file join $src_project_root src]
#src/lib and src/lib_tcl<tclmajor> (editable library source)
foreach libsub [list lib_tcl$tclmajorv lib] {
set libfolder [file join $src_lib_base $libsub]
if {[file isdirectory $libfolder]} {
if {[string match lib_tcl* [file tail $libfolder]]} {
if {[file exists $libfolder/allplatforms]} {
lappend auto_path_additions_src $libfolder/allplatforms
}
if {[file exists $libfolder/$platform]} {
lappend auto_path_additions_src $libfolder/$platform
}
} else {
lappend auto_path_additions_src $libfolder
}
}
}
#src/bootsupport/lib (bootstrap libraries)
set src_bs_lib [file join $src_lib_base bootsupport lib]
if {[file isdirectory $src_bs_lib]} {
if {$src_bs_lib ni $auto_path_additions_src} {
lappend auto_path_additions_src $src_bs_lib
}
}
#src/bootsupport/lib/tcl<tclmajor>/<arch> (platform-specific bootstrap libraries)
set src_bs_lib_arch [file join $src_lib_base bootsupport lib tcl$tclmajorv $platform]
if {[file isdirectory $src_bs_lib_arch]} {
if {$src_bs_lib_arch ni $auto_path_additions_src} {
lappend auto_path_additions_src $src_bs_lib_arch
}
}
}
# -- --- --- --- --- --- --- ---
#split existing ::auto_path entries into internal & external
set internal_ap_dirs [list] ;#
@ -743,6 +839,13 @@ apply { args {
}
}
}
src {
foreach n $auto_path_additions_src {
if {$n ni $new_auto_path} {
lappend new_auto_path $n
}
}
}
os {
foreach n $external_ap_dirs {
if {$n ni $new_auto_path} {
@ -900,6 +1003,84 @@ apply { args {
#assert arglist has had 'dev|os|os-dev etc' first arg removed if it was present.
#--------------------------------------------------------
#src mode: register #modpod modules from src/modules via package ifneeded
#and set package prefer latest so 999999.0a1.0 dev modules are preferred
#over stable bootsupport/vendored copies on unversioned package require.
#Uses only Tcl builtins (glob, file, string) since no punk modules are loaded yet.
#--------------------------------------------------------
if {"src" in $package_modes && [info exists src_project_root] && $src_project_root ne ""} {
#package prefer latest is set here so 999999.0a1.0 dev modules are preferred
#over stable bootsupport/vendored copies on unversioned package require.
#This must be set after libunknown::init (which may reset the preference to stable)
#and before any package require calls in the subcommand handler below.
#We set it again just before subcommand dispatch to ensure it isn't overridden.
#inline #modpod scanner — equivalent to punk::tcltestrun::tm_path_additional_ifneeded
#but using only Tcl builtins since punk::path isn't loaded at boot time.
set src_modules_dir [file join $src_project_root src modules]
set modpod_count 0
if {[file isdirectory $src_modules_dir]} {
#recursive glob for #modpod-* directories (Tcl 8.6+ supports ** in glob)
set modpod_dirs [list]
foreach found [glob -nocomplain -type d -directory $src_modules_dir ** #modpod-*] {
#skip _build subdirectories
if {[string match "*_build*" $found]} { continue }
lappend modpod_dirs $found
}
foreach modpod_dir $modpod_dirs {
set tail [file tail $modpod_dir]
#directory name format: #modpod-<modname>-<version>
#strip leading "#modpod-" (8 chars), then split on last "-" to separate modname from version
set rest [string range $tail 8 end]
set last_dash [string last "-" $rest]
if {$last_dash < 0} { continue }
set modname [string range $rest 0 [expr {$last_dash - 1}]]
set modver [string range $rest [expr {$last_dash + 1}] end]
set modpath [file join $modpod_dir "$modname-$modver.tm"]
#compute fully qualified module name from path relative to src/modules
#file relative isn't available in all Tcl builds at boot time, so compute manually
set reldir ""
set checkdir $modpod_dir
set base $src_modules_dir
#walk up from modpod_dir until we reach src/modules, collecting path components
while {$checkdir ne $base && $checkdir ne ""} {
set reldir [linsert $reldir 0 [file tail $checkdir]]
set checkdir [file dirname $checkdir]
}
if {$reldir eq ""} {
set fullmodname $modname
} else {
set fullmodname [join $reldir ::]::$modname
}
if {[file exists $modpath]} {
eval [list package ifneeded $fullmodname $modver [list source $modpath]]
incr modpod_count
}
}
if {$modpod_count > 0} {
puts stderr "src mode: registered $modpod_count #modpod module[expr {$modpod_count == 1 ? "" : "s"}] from $src_modules_dir"
}
}
}
#--------------------------------------------------------
#src mode: set package prefer latest as late as possible (after libunknown::init
#which may reset it to stable) so 999999.0a1.0 dev modules are preferred over
#stable bootsupport/vendored copies on unversioned package require.
if {"src" in $package_modes} {
package prefer latest
# Force a scan of all tcl::tm::list paths by triggering package unknown.
# The VFS-bundled stable versions (e.g punk 0.1.1) are already registered
# from Tcl's init, so package require for those packages would never call
# package unknown — meaning 999999.0a1.0 dev versions in src/modules would
# never be discovered. This dummy require forces package unknown to scan
# all tm paths and register all ifneeded scripts, including the dev versions.
# After this, package prefer latest will select 999999.0a1.0 over 0.1.1.
catch {package require __src_mode_tm_scan__}
}
#tclsh,shellspy,punk,script,shell
set subcommand [lindex $arglist 0]
switch -- $subcommand {

Write Preview
Loading…
Cancel
Save