5 goaljobs-reference - reference documentation for writing goaljobs scripts
13 let goal name args... =
17 (* code to implement the goal *)
20 require (name args...)
29 Goaljobs is a flexible build system and business rules manager similar
30 to make and cron, but much more powerful. You can use it to automate
31 many complex tasks that have multiple steps (even with manual steps)
32 that have to be carried out in dependency order.
34 For a tutorial-like introduction to goaljobs, see
35 L<http://rwmj.wordpress.com/tag/goaljobs/>
37 For examples, see the C<examples/> directory in the source and
38 L<http://git.annexia.org/?p=goals.git;a=summary>
40 For reference documentation on how to write scripts, see below.
42 Note this man page does not cover the whole Goaljobs API. To read
43 about the Goaljobs API, look for the file C<goaljobs.mli> (in the
44 source code or installed as part of the goaljobs package), or see the
45 HTML documentation installed as part of the goaljobs package.
47 =head1 THE SCRIPT FILE
49 The script file should usually start with opening these modules (none
50 of this are required, it's just useful to have them open):
56 This is followed by goals and/or functions and/or top-level OCaml
57 statements and/or C<every> statements (a.k.a periodic jobs).
59 You can use multiple script files to make up a goaljobs program. You
60 have to list them in dependency order on the goaljobs command line
61 (earlier files required by later files), the same way that the OCaml
62 compiler works. So usually you end up writing:
64 goaljobs utils.ml another_library.ml script.ml
66 where C<script.ml> requires the functions from the utils/library.
67 Note that circular dependencies are not possible.
71 Each goal should have the following basic form:
73 let goal name args... =
77 (* code to implement the goal *)
79 There is no hard-and-fast rule about this. In particular you can put
80 arbitrary OCaml statements anywhere inside a goal (since a goal is
81 just a special form of OCaml function), but sticking to this overall
84 There should be zero or one target. Multiple target statements should
85 not be used in a goal. The target should come as early as possible,
86 and the target condition should be as simple and fast to evaluate as
87 is practical (see L</THE MEMORY> below).
89 There should be zero or any number of C<require> statements. Each
90 require statement should name a single goal (with optional parameters
93 After that should come the code that implements the goal, which might
94 be, for example, a series of shell commands, but could even be
97 As with ordinary OCaml functions, you can define goals recursively
98 or with mutual recursion using:
100 let rec goal1 args... =
107 A goal can also have no arguments:
112 This defines the common goal called C<all>, which acts the same way as
113 C<make all>, ie. if you run the program without any arguments, it will
114 run the C<all> goal if one exists.
116 =head2 PUBLISHING GOALS
118 If a goal is "published" it means it is available to be run directly
119 from the command line. All no-arg goals are published by default.
120 You do not need to do anything special for them. For example:
122 let goal clean () = sh "rm *~"
124 can be used on the command line:
128 For goals which take any parameters, you have to define a small code
129 snippet that converts command line arguments to goal parameters (the
130 reason has to do with OCaml being strongly typed, and because goal
131 parameters might not all be strings).
133 let goal compile program sources =
134 target (more_recent [program] sources);
140 let program = List.hd args in
141 let sources = List.tl args in
142 require (compiled program sources)
147 ./myscript compile program main.c utils.c
149 =head1 TARGET AND REQUIRE
151 The target is promise or contract that you make that the given
152 condition I<will> be true when the goal has finished running.
154 In the first example, the target is that the C<o_file> (object) exists
155 and is newer than the C<c_file> (source). The goal meets that target
156 by running the C compiler (C<cc>) which, if it succeeds, will ensure
157 that the object file exists and is newer than the source file.
159 let goal compiled c_file =
160 let o_file = change_file_extension "o" c_file in
161 target (more_recent [o_file] [c_file]);
168 In the second example, the goal requires that several files have been
169 compiled (C<require (compiled ...)>) before it can link the final
172 let goal built program source =
173 target (more_recent [program] [source]);
175 require (compiled source);
177 let object = change_file_extension "o" source in
183 =head1 SPECIAL VALUES INSIDE GOALS
187 Inside goals, you can use C<goalname> to get the name of the goal, ie:
190 printf "my name is %s\n" goalname
198 Inside goals, you can use C<goalloc> to get a printable source
199 location of the goal, ie:
202 printf "%s\n" goalloc
206 File "source.ml", line 2, characters 13-71 (end at line 3, character 23)
208 Note that the actual string format depends on the internal OCaml
209 function C<Loc.to_string> so it might change in future.
211 =head2 onfail, onsuccess, onrun
213 Inside goals you can register function(s) which run if the goal
214 completes successfully (C<onsuccess>), if the goal completes
215 successfully after running to the end (C<onrun>), or if the goal fails
221 onfail (fun _ -> eprintf "goal '%s' failed\n" goalname);
226 If the shell command (or another part of the goal) fails, then this
231 The single parameter passed to C<onfail> is the exception that was
234 Note that the helper function C<Goaljobs.mailto> is a useful function
235 to call from an C<onfail> handler:
237 let from = "me@example.com"
238 let to_ = "you@example.com"
239 let logfile = log_program_output ()
243 let subject = sprintf "goal: %s: BUILD FAILED" goalname in
244 mailto ~from ~subject ~attach:[logfile] to_);
249 C<onsuccess> and C<onrun> are slightly different from C<onfail> and
252 C<onsuccess> functions can be called if a C<target> condition is met
253 and the rest of the goal is short-circuited. C<onrun> will only be
254 called if all the instructions in the goal actually run and succeed.
256 The single unit C<()> parameter is passed to the C<onsuccess> and
259 You can register as many functions as you want for each handler. The
260 order in which the functions are called is not defined.
264 If you want to have a goal that runs when some outside event happens
265 you have three choices: Manually run the script (this is basically
266 what C<make> forces you to do). Have some sort of hook that runs the
267 script (eg. a git hook). Or use a periodic job to poll for an event
270 Periodic jobs run regularly to poll for an outside event or change.
271 If a script has periodic jobs, then it runs continuously (or until you
274 An example of a script that checks for new git commits and when it
275 sees one it will ensure it passes the tests:
277 let repo = Sys.getenv "HOME" // "repo"
279 let goal git_commit_tested commit =
280 let key = sprintf "repo-tested-%s" commit in
281 target (memory_exists key);
291 (* Record that this commit was tested successfully. *)
294 every 30 minutes (fun () ->
295 let commit = shout "cd %s && git rev-parse HEAD" repo in
296 (* Require that this commit has been tested. *)
297 require (git_commit_tested commit)
300 Some notes about the above example: Firstly only the current HEAD
301 commit is required to be tested. This is because older commits are
302 irrelevant and because if they failed the test before there is not
303 point retesting them (commits are immutable). Secondly we use the
304 Memory to remember that we have successfully tested a commit. This is
305 what stops the program from repeatedly testing the same commit.
309 You can call out to the Unix shell using one of the functions
310 C<Goaljobs.sh>, C<Goaljobs.shout> or C<Goaljobs.shlines>. (These
311 functions are documented in the C<goaljobs.mli> file / API
314 C<sh> runs the command(s). C<shout> collects the output of the
315 command (to stdout only) and returns it as a single string.
316 C<shlines> collects the output and returns it as a list of lines.
318 C<sh>, C<shout>, C<shlines> work like printf. ie. You can substitute
319 variables using C<%s>, C<%d> and so on. For example:
321 sh "rsync foo-%s.tar.gz example.com:/html/" version
323 Each shell runs in a new temporary directory. The temporary directory
324 and all its contents is deleted after the shell exits. If you want to
325 save any data, C<cd> somewhere. If you don't want the temporary
326 directory creation, use C<~tmpdir:false>.
328 The environment variable C<$builddir> is exported to the script. This
329 is the current directory when the goaljobs program was started.
331 Each invocation of C<sh> (etc) is a single shell (this is slightly
332 different from how C<make> works). For example:
336 tarball=$package.tar.gz
344 The shell error mode is set such that if any single command
345 returns an error then the C<sh> function as a whole exits with
350 to ignore the result of a command.
352 C</bin/sh> is used unless you set C<Goaljobs.shell> to some other
353 value. Note that the environment variable C<SHELL> is I<never> used.
357 "The Memory" is key/value storage which persists across goaljobs
358 sessions. It is stored in the file C<$HOME/.goaljobs-memory> (which is
359 a binary file, but you can delete it if you want).
361 The Memory is locked during accesses, so it is safe to read or write
362 it from multiple parallel goaljobs sessions.
364 Keys and values are strings. The keys should be globally unique, so
365 it is suggested you use some application-specific prefix. eg:
370 let goal tested version =
371 let key = "myapp-tested-" ^ version in
372 target (memory_exists key);
374 (* some code to test this version *)
378 Note in that example the value C<1> is arbitrary. You just want to
379 store I<any> value so that a later call to C<memory_exists> will
382 For information about C<Goaljobs.memory_*> APIs see the
383 C<goaljobs.mli> file / API documentation.
391 This is the default shell used by C<sh*> APIs. You can change
392 the shell by setting the C<Goaljobs.shell> reference.
396 The curl program (on the path) is used to check for and download
397 URLs by APIs such as C<Goaljobs.url_exists>.
399 =item C<~/.goaljobs-memory>
401 Persistent key/value store used when you use the C<Goaljobs.memory_*>
406 =head1 ENVIRONMENT VARIABLES
412 This environment variable is set to the current directory when the
413 goals program starts, and is available in goals, shell scripts, etc.
423 Richard W.M. Jones <rjones @ redhat . com>
427 (C) Copyright 2013 Red Hat Inc.,
429 This program is free software; you can redistribute it and/or modify
430 it under the terms of the GNU General Public License as published by
431 the Free Software Foundation; either version 2 of the License, or
432 (at your option) any later version.
434 This program is distributed in the hope that it will be useful,
435 but WITHOUT ANY WARRANTY; without even the implied warranty of
436 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
437 GNU General Public License for more details.
439 You should have received a copy of the GNU General Public License
440 along with this program; if not, write to the Free Software
441 Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.