google/go-cmdtest: This package simplifies testing of command-line interfaces.
google/go-cmdtest: This package simplifies testing of command-line interfaces.
google/go-cmdtest: This package simplifies testing of command-line interfaces.
rendon/testcli: Go lang testing framework for command line applications.
rendon/testcli: Go lang testing framework for command line applications.
lucapette/go-cli-integration-tests
lucapette/go-cli-integration-tests
- Its README is out-of-date and has an old reference to a package that had its import path changed (easily fixed)
- Running the tests failed miserably as it could not find the
greeting binary in the $PATH π€¦ββοΈda fuq?! I guess this doesn't do what I _thought_ -- which is to build the test binary and use that to run CLI tests against so you can actually measure coverage π
Next!
- Its README is out-of-date and has an old reference to a package that had its import path changed (easily fixed)
- Running the tests failed miserably as it could not find the
greeting binary in the $PATH π€¦ββοΈda fuq?! I guess this doesn't do what I _thought_ -- which is to build the test binary and use that to run CLI tests against so you can actually measure coverage π
Next!
This is eventually what I came up with so far... What do you think @lyse ? π€
This is eventually what I came up with so far... What do you think @lyse ? π€
My
main()s often also just do os.Exit(run()). Passing the command line arguments run run(β¦) seems like another obvious thing to do. Even though, I should have done this more consistently, I reckon. I feel very stupid now, because for whatever reason, it never occurred to me to simply pass an io.Writer for stdout. I really like that. Although, I'm wondering why there's nothing for stderr. Errors should definitely not mix with other output in my opinion. Anyways. When testing, I always captured stdout with a much more complicated code segment so far. Store the original output and error streams, set the new ones, execute the test, convert things to strings and finally reset to the original streams. I will definitely adopt these io.Writer arguments. Thanks, mate! :-)This cmdtest test execution also captures the coverage? It looks kinda more complicated than it should be, otherwise. Just a program with the test definition file would be enough in my opinion.
I don't know if I like the concept of providing a single test definition file or not. It's a bit intriguing, but I fear that's not flexible enough. Just a gut feeling, might be wrong.
cmdtest package is kind of cool though really, it basically implements the same kind of test runner as you may (or may not) have seen in the Mercurial test suite. The test files in testdata are essentially text files that look a bit like you've run something on the console and copied pasted them. This is brilliant for e2e cli integration testing π And yes it manages to run the test binary so that coverage can also be measured which is fantastic π -- Of course this does not preclude you from writing unit tests for any other parts of your package/library that have a public facing API -- But if your public facing API is _just_ the CLI then this is a perfect fit π
cmdtest package is kind of cool though really, it basically implements the same kind of test runner as you may (or may not) have seen in the Mercurial test suite. The test files in testdata are essentially text files that look a bit like you've run something on the console and copied pasted them. This is brilliant for e2e cli integration testing π And yes it manages to run the test binary so that coverage can also be measured which is fantastic π -- Of course this does not preclude you from writing unit tests for any other parts of your package/library that have a public facing API -- But if your public facing API is _just_ the CLI then this is a perfect fit π
io.Writer is left out for "stderr" is that normally I tend to just set the logging output to os.Stderr, like log.SetOutput(os.Stderr) anyway and its not usually something I end up testing. Not sure if this is the best approach, but I'm only really interested in testing the "output"(s) and either error or non-error cases.
io.Writer is left out for "stderr" is that normally I tend to just set the logging output to os.Stderr, like log.SetOutput(os.Stderr) anyway and its not usually something I end up testing. Not sure if this is the best approach, but I'm only really interested in testing the "output"(s) and either error or non-error cases.
go-cmdtest merges both stdout and -err, that's a no-go in my books.
Main() has the func Main(w io.Writer, args []string) error π
So you _can_ actually assert on the error returned. For a CLI however, I'm not particularly a fan of logging errors to stderr too much (if at all). And re go-cmdtest FWIW a Terminal combines stdout and stderr too by default when displaying the output of a program π
-- However I filed an issue against the cmdtest project and now I'm not so sure I want to continue using it, I _may_ as well just figure out how to run the test binary under coverage and write the tests myself in Go.
Main() has the func Main(w io.Writer, args []string) error π
So you _can_ actually assert on the error returned. For a CLI however, I'm not particularly a fan of logging errors to stderr too much (if at all). And re go-cmdtest FWIW a Terminal combines stdout and stderr too by default when displaying the output of a program π
-- However I filed an issue against the cmdtest project and now I'm not so sure I want to continue using it, I _may_ as well just figure out how to run the test binary under coverage and write the tests myself in Go.
error return value! That sounds good to me. When there are just fatal errors that abort the program execution, a main function returning an error is definitely enough.Hmm, if you don't want to report errors to stderr, where do you write them to? Hopefully not stdout. A log file? It obviously depends on the program and such, but generally I do not want to dig up errors from a log file. Usually, I find it much more convenient to see them directly. Properly dealing with stdout and stderr basically provides the capabilities for free to be pipeline-ready. And of course,
-q or something along those lines is also a good choice. When talking about more serious programs, that is. Not just some quickly cobbled together helper.