1 TITLE: "goals" is a new tool which generalizes "make"
3 This talk is *not* about several things. It's *not* about build
4 tools. It's *not* about how autoconf sucks or the best tool
5 to build Java software. It's *not* about package management,
6 continuous integration, package ecosystems or anything like that.
8 It's about one tool which is over 40 years old: MAKE. Designed by
9 Stuart Feldman in 1976.
15 - Others are possible,
16 eg: URL, newer than any file (not all files),
17 Koji build, comparing checksums, test with skip
20 - "all" is not like a file
22 SINGLE PARAMETER PROBLEM:
23 - %.o: %c only allows a single parameter
26 - How do you quote a file with spaces?
27 https://stackoverflow.com/questions/15004278/allow-space-in-target-of-gcc-makefile
28 For a tool whose main job is running shell commands
29 it has quite a lot of problems with shell commands:
34 - Invisible whitespace is meaningful
35 - Individual commands are passed to separate shells
36 - $ has meaning to both shell and make
41 Let's talk about shell scripting first, because that's easiest to fix:
43 target: foo.o bar.o "target": "foo.o", "bar.o" {
44 $CC $CFLAGS $< -o $@ %CC %CFLAGS %< -o %@
47 The new tool uses a real LALR(1) parser. Filenames have to be
48 enclosed in quotes, code always appears in curly braces, % is
49 the magic character for variables leaving $ for the shell to use,
50 commands in the code block run under a single shell (but any
51 command returning an error is still fatal), and the formatting
52 is free-form so there's no special whitespace.
56 Goals can optionally be given names and parameters:
61 "target" : "foo.o", "bar.o" { ... }
64 "%name.o" : "%name.c", "dep.h" { %CC %CFLAGS -c $^ -o $@ }
67 You can run a goal in two ways. The "make way" is to find
68 the target that matches the given filename. "foo.o" matches
69 "%name.o" and so we know to run the compile goal. But
70 if you want you can also run compile ("bar") directly:
75 "target" : "foo.o", compile ("bar") { ... }
78 "%name.o" : "%name.c", "dep.h" { %CC %CFLAGS -c $^ -o $@ }
82 Tactics are special rules that we can use to change how we
83 determine if a goal needs to be rebuilt. When you see a
84 filename string, there's an implicit tactic called *file, so
85 these are equivalent, because when goals sees a bare string
86 but it wants a tactic it implicitly uses *file.
88 "target" : "foo.o", "bar.o" { ... }
90 *file("target") : *file("foo.o"), *file("bar.o") { ... }
93 Apart from *file being the default tactic, it's not built
94 into goals. In fact *file is defined in the goals standard
95 library. The special @{...} code section means the code
96 doesn't print verbosely when its running. And "exit 99"
97 is used by the tactic to indicate that the target needs
98 to be rebuilt, but other than that it's all written in
99 ordinary shell script:
101 tactic *file (filename) = @{
102 test -f %filename || exit 99
104 test %filename -ot "$f" && exit 99 ||:
110 And you can of course write other tactics in shell script.
111 Here's a tactic for running test suites. This tactic lets
112 you skip a test by setting an environment variable.
114 tactic *test (script) = @{
115 # Check if SKIP variable is set.
117 echo -n SKIP_%script |
121 if test "${!skip_var}" = "1"; then exit 0; fi
122 if test %goals_final_check; then exit 0; else exit 99; fi
125 You can use the tactic like this. There's quite a lot to unpack
126 in this example, but I'll just say that the wildcard function
127 expands to a list of files, and the wrap function changes them
128 from a list of strings into a list of *test tactics.
130 let tests = wrap ("*test", wildcard ("test-*.sh"))
131 goal check () = : tests
132 goal test (script) = *test(name) : { ./%name }
135 Another tactic we use is called *built-in-koji, which I
136 use for mass rebuilding Fedora packages in dependency order.
137 I won't go into the full definition of *built-in-koji since
138 interfacing with Koji is quite complicated, but you can
139 write a mass rebuild tool in goals fairly easily:
141 [SHOW OUTLINE FROM fedora-ocaml-rebuild/Goalfile]
144 We saw a couple of standard functions in the test example -
145 "wildcard" and "wrap". In make there are many built in functions.
146 In goals, all functions are defined in a standard library and
147 written in the goals language plus shell script. Here's the
148 definition of the wildcard function, this is the actual
149 code you're running if you use the wildcard function in a
150 Goalfile. You can see that functions can take zero, one, or
151 more parameters, and they can return strings or arbitrary Goalfile
154 function wildcard (wc) returning strings = @{
157 for f in $wc; do echo "$f"; done
161 In fact goals consists of a native core language parser and runtime
162 for evaluating the language, building the dependency graph and
163 executing jobs in parallel. But around this small core is a
164 large standard library which is written in the goals language
165 plus shell script. So in a sense goals is bootstrapped from
166 a smaller core into a larger ecosystem, which makes it quite
167 different from "make".
171 COMPUTER SCIENCY THINGS
172 (not necessary to know this)
174 There are some interesting parallels between the goals language
175 and programming languages that I want to highlight. Not least
176 because they point to future ways we might explore this space.
179 - Goals are functions + dependency solving
181 goal clean () = { rm -f *~ }
183 goal all () = : "program"
185 goal link = "program" : "foo.o" { %CC %CFLAGS %< -o %@ }
188 - Tactics are constructors
189 - Targets are patterns
191 *file ("%name.o") : ... match name with
192 | File (name + ".o") -> compile name
195 - But our pattern matcher is very naive, could it be more complex?
196 What would that mean?
197 SCREENSHOT OF ZINC PAPER
200 - Goal "functions" may be called by name or by pattern,
201 which is unusual. Is there another programming language
206 - Dependencies have implicit & operator, could we use | and ! operators?
207 What would that mean? Build targets in several different ways?
208 Fallback if a tool isn't available?
214 - Types other than strings. Int and bool would be useful. However
215 this also implies that we should do type inference and/or checking.
219 goal build (project, bool release = true) = ...
223 - Anonymous functions
225 let hello = { echo "hello" }
227 let f = function (name, version) { CODE }