X-Git-Url: http://git.annexia.org/?p=libguestfs-talks.git;a=blobdiff_plain;f=2020-goals%2Fnotes.txt;h=b03d9e5ea1062869c642a1850a09340ea1575116;hp=68604c68ab30c3c48777cea28e43c7f48812c5aa;hb=HEAD;hpb=92a5a3f603374f88037129ac1a1456a2c250c4ef diff --git a/2020-goals/notes.txt b/2020-goals/notes.txt index 68604c6..e76dff1 100644 --- a/2020-goals/notes.txt +++ b/2020-goals/notes.txt @@ -8,40 +8,86 @@ continuous integration, package ecosystems or anything like that. It's about one tool which is over 40 years old: MAKE. Designed by Stuart Feldman in 1976. +Make is a great tool! It's easy to get started, intuitive, +and wildly successful. If I critize make it's not because I +think it's a bad tool, just that we could do even better if +we addressed some shortcomings. -TACTIC PROBLEM: - - Only one tactic. - - Others are possible, + + +PREDICATE PROBLEM: + +Only one predicate for solving dependencies. + +"If the target file doesn't exist, or if it's older than +one of the dependencies, then run this recipe." + +If you think for a little while you'll see that other +predicates are possible: + eg: URL, newer than any file (not all files), Koji build, comparing checksums, test with skip + + PHONY FILE PROBLEM: - - "all" is not like a file + +"test" is not a file. + +If "test" happens to be created, your tests will stop running +silently. + +Experienced users know they can use the .PHONY directive to +avoid this. + +Make doesn't check that when you run a rule that it actually +creates the file that it says it creates. +The new tool called goals does check this. + +It also points to a fundamental issue: the target +is overloaded to mean either a file or a rule name. + + SINGLE PARAMETER PROBLEM: - - %.o: %c only allows a single parameter + +Make recipes can be parameterized, provided you only need +a single parameter. + +If a single parameter is useful, it's not outlandish to +imagine that having two parameters could be useful, or +even more. + + + SHELL PROBLEM: - - How do you quote a file with spaces? - https://stackoverflow.com/questions/15004278/allow-space-in-target-of-gcc-makefile - For a tool whose main job is running shell commands - it has quite a lot of problems with shell commands: + +How do you quote a file with spaces? +https://stackoverflow.com/questions/15004278/allow-space-in-target-of-gcc-makefile + +For a tool whose main job is running shell commands +it has quite a lot of sharp edges when running shell commands. + dest/target: deps cd dest var=1 echo $var > target - - Invisible whitespace is meaningful + - Individual commands are passed to separate shells + - $ has meaning to both shell and make + - Invisible whitespace is meaningful + Let's talk about shell scripting first, because that's easiest to fix: target: foo.o bar.o "target": "foo.o", "bar.o" { - $CC $CFLAGS $< -o $@ %CC %CFLAGS %< -o %@ + ${CC} ${CFLAGS} $< -o $@ %CC %CFLAGS %< -o %@ } The new tool uses a real LALR(1) parser. Filenames have to be @@ -79,26 +125,26 @@ if you want you can also run compile ("bar") directly: -Tactics are special rules that we can use to change how we +Predicates are special rules that we can use to change how we determine if a goal needs to be rebuilt. When you see a -filename string, there's an implicit tactic called *file, so +filename string, there's an implicit predicate called is-file, so these are equivalent, because when goals sees a bare string -but it wants a tactic it implicitly uses *file. +but it wants a predicate it implicitly uses is-file. "target" : "foo.o", "bar.o" { ... } - *file("target") : *file("foo.o"), *file("bar.o") { ... } + is-file("target") : is-file("foo.o"), is-file("bar.o") { ... } -Apart from *file being the default tactic, it's not built -into goals. In fact *file is defined in the goals standard +Apart from is-file being the default predicate, it's not built +into goals. In fact is-file is defined in the goals standard library. The special @{...} code section means the code doesn't print verbosely when its running. And "exit 99" -is used by the tactic to indicate that the target needs +is used by the predicate to indicate that the target needs to be rebuilt, but other than that it's all written in ordinary shell script: - tactic *file (filename) = @{ + predicate is-file (filename) = @{ test -f %filename || exit 99 for f in %<; do test %filename -ot "$f" && exit 99 ||: @@ -107,11 +153,11 @@ ordinary shell script: -And you can of course write other tactics in shell script. -Here's a tactic for running test suites. This tactic lets +And you can of course write other predicates in shell script. +Here's a predicate for running test suites. This predicate lets you skip a test by setting an environment variable. - tactic *test (script) = @{ + predicate is-test (script) = @{ # Check if SKIP variable is set. skip_var=$( echo -n SKIP_%script | @@ -122,23 +168,24 @@ you skip a test by setting an environment variable. if test %goals_final_check; then exit 0; else exit 99; fi } -You can use the tactic like this. There's quite a lot to unpack +You can use the predicate like this. There's quite a lot to unpack in this example, but I'll just say that the wildcard function expands to a list of files, and the wrap function changes them -from a list of strings into a list of *test tactics. +from a list of strings into a list of is-test predicates. - let tests = wrap ("*test", wildcard ("test-*.sh")) + let tests = wrap ("is-test", wildcard ("test-*.sh")) goal check () = : tests - goal test (script) = *test(name) : { ./%name } + goal run (script) = is-test(script) : { ./%script } -Another tactic we use is called *built-in-koji, which I +Another predicate we use is called is-koji-built, which I use for mass rebuilding Fedora packages in dependency order. -I won't go into the full definition of *built-in-koji since +I won't go into the full definition of is-koji-built since interfacing with Koji is quite complicated, but you can write a mass rebuild tool in goals fairly easily: -[SHOW OUTLINE FROM fedora-ocaml-rebuild/Goalfile] + [ DISCUSSION OF FEDORA-OCAML-REBUILD IN SLIDES ] + We saw a couple of standard functions in the test example - @@ -185,24 +232,69 @@ because they point to future ways we might explore this space. goal link = "program" : "foo.o" { %CC %CFLAGS %< -o %@ } - - Tactics are constructors + - Predicates are constructors - Targets are patterns - *file ("%name.o") : ... match name with + is-file ("%name.o") : ... match name with | File (name + ".o") -> compile name | ... - - But our pattern matcher is very naive, could it be more complex? - What would that mean? -SCREENSHOT OF ZINC PAPER - - - Goal "functions" may be called by name or by pattern, which is unusual. Is there another programming language which does this? (Prolog actually) + - But our pattern matcher is very naive, could it be more complex? + What would that mean? + + SCREENSHOT OF ZINC PAPER + + - Dependencies have implicit & operator, could we use | and ! operators? What would that mean? Build targets in several different ways? Fallback if a tool isn't available? + + + +TO-DO + + - Types other than strings. Int and bool would be useful. However + this also implies that we should do type inference and/or checking. + + - Default arguments + + goal build (project, bool release = true) = ... + build ("foo") + build ("foo", false) + + - Anonymous functions + Any code section is potentially a closure + + let hello = { echo "hello" } + + let f = function (name, version) { CODE } + f ("goals", "0.1") + + + - Cover some of the other tools that are better makes. + (Note I'm not covering all DAG solvers like Ninja unless they + bring something new to the table) + + * Plan 9 mk - has alternate build predicates + + * https://github.com/casey/just - has functions and multi-parameter rules + + * http://www.crufty.net/help/sjg/bmake-meta-mode.htm + - alternative route to finding deps by snooping on file + accesses at the kernel level + * https://github.com/apenwarr/redo - similar to above + + * https://web.archive.org/web/20001027183954/http://software-carpentry.codesourcery.com/entries/build/Tromey/Tromey.html + + +---- +to do: + + - Explain predicates better, Kashyap found it confusing. +