Commit Graph

44 Commits

Author SHA1 Message Date
Arseny Kapoulkine ba65b0e9cd
Build and use luau-compile in CI (#952) 2023-06-12 08:46:44 -07:00
Arseny Kapoulkine 29d0ea10cf
Add DCR solver to analysis benchmarks (#862)
For now just do this in strict mode. This will help us track performance
over time, although for now the behavior is going to keep changing so
it's not going to be a fully solid metric for a few weeks.
2023-03-13 13:58:29 -07:00
vegorov-rbx 685ca02a30
Keep using ubuntu-20.04 in GitHub Actions (and clang++-10 in coverage) (#795)
### Problem
ubuntu-latest was updated to 22.04 which removes clang++-10 we used for
coverage stats and creates a pre-compiled binary that requires a glibc

### Solution
Pin to ubuntu-20.04 using multi-value matrix configurations.
In coverage configuration, we use clang++-10 once again.
2023-01-06 12:17:25 -08:00
vegorov-rbx 91302d1d4c
Fix coverage github action by using clang++ without a specific version (#769)
Github fails to find clang++-10.
Since we install llvm without a specific version number, we shouldn't
use clang++-10 with a version number.

I tried installing llvm-10, but that package is not available on github
(any more?).
2022-12-02 10:08:55 -08:00
Arseny Kapoulkine e64d489e57
Update benchmark.yml
Add codegennull runs to compile time benchmarks
2022-10-24 17:24:19 -07:00
Arseny Kapoulkine 49d6bc30ad
Add bench-codegen benchmark (#713)
We will now run luau with --codegen during benchmark runs and collect
the data into separate JSON. Note that we don't yet have the historical
data for these, which will be backfilled later.
2022-10-17 12:18:34 -07:00
Arseny Kapoulkine c6a2d75193
Improve testing coverage with codegen/O2 in mind (#709)
This change adds codegen runs to coverage config and adds O2/codegen
testing to CI.

Note that we don't run O2 combinations in coverage - it's better that we
see gaps in O2 coverage in compiler tests, as these are valuable for
validating codegen intricacies that are difficult to see from
conformance tests passing/failing.
2022-10-17 10:02:21 -07:00
Arseny Kapoulkine d6aa35583e Add new-release action to automatically build & publish release artifacts 2022-10-06 18:44:51 -07:00
Arseny Kapoulkine cc26ef16df
Update release.yml
Use -j2 for release builds since CMake doesn't do it for us
2022-09-29 16:19:37 -07:00
Arseny Kapoulkine 937ef2efd4
Cleanup benchmark builds a little bit (#691)
We don't need to run any cachegrind benchmarks in benchmark-dev, since
benchmark uses our new callgrind setup instead.

Also removes prototyping filters that we no longer need from all builds.
2022-09-29 15:42:23 -07:00
Arseny Kapoulkine 1acd66c97d
Remove prototyping/ and prototyping.yml (#688)
Superseded by
2022-09-29 12:13:00 -07:00
Arseny Kapoulkine 8b36409c2c
Update build.yml (#675)
Pass codecov token explicitly, for some reason automatic configuration
stopped working
2022-09-16 11:39:30 -07:00
Allan N Jeremy 366df96393
Feat: Added retry functionality to benchmarks (#667)
Resolves #668 

## The problem
Benchmarks jobs run concurrently for the different operating systems.
This means that when it comes time to push the benchmark results to [the
assigned benchmark results
repo](, there can be two
different jobs trying to push changes at the same time. In such a case,
one of the pushes will fail and we end up missing some benchmark results
data from the workflow run.

## The solution
Whenever a push fails, we need to retry the steps leading up to the push
(checking out the benchmark results repo, storing benchmark results,
pushing the results to [a specific

### Note
There are 3 push attempts before submitting to failure.

## TL;DR
This PR retries pushing benchmark results when they fail to get pushed
(often due to pushing from multiple jobs concurrently)

Co-authored-by: Jamie Kuppens <>
Co-authored-by: Ignacio Falk <>
2022-09-12 08:45:55 -07:00
Arseny Kapoulkine 8b390a33aa
Update benchmark.yml
Switch to bench.json file so that bench-gcc is the only outlier, as everything else is built with clang.
2022-08-11 14:14:32 -07:00
Arseny Kapoulkine f7d8ad0774
Sync to upstream/release/540 (#635)
Also adjust benchmark runs to use config=profile and run clang for all benchmarks + gcc for runtime
2022-08-11 14:01:33 -07:00
Arseny Kapoulkine a824b05c9e
Update coverage report to codecov (#606)
Coveralls has had multiple stability issues in the recent while, and codecov seems much better maintained in general.
2022-07-20 15:12:30 -07:00
Allan N Jeremy dbcd5fb28e
build: changed data output to be different for each os (#581)
- macos: data-macos-latest.json
- ubuntu: data-ubuntu-latest.json
- windows: data-windows-latest.json
2022-07-07 08:21:40 -07:00
Arseny Kapoulkine 2460e38998
bench: Implement luau-analyze and luau --compile benchmarks (#575)
This change adds another file for benchmarking luau-analyze and sets up
benchmarks for both non-strict/strict modes for analysis and all three
optimization levels for compilation performance.

To avoid issues with race conditions on repository update we do all this
in the same job in benchmark.yml.

To be able to benchmark both modes from a single file, luau-analyze
gains --mode argument which allows to override the default typechecking
mode. Not sure if we'll want this to be a hard override on top of the
module-specified mode in the future, but this works for now.
2022-07-05 08:23:09 -07:00
Arseny Kapoulkine 48aa7a5162
bench: Implement first class support for callgrind (#570)
Since callgrind allows to control stats collection from the guest, this
allows us to reset the collection right before the benchmark starts.

This change exposes this to the benchmark runner and integrates
callgrind data parsing into, so that we can run with
--callgrind argument and, as long as the runner was built with callgrind
support, we get instruction counts from the run.

We convert instruction counts to seconds using 10G instructions/second
rate; there's no correct way to do this without simulating the full CPU
pipeline but it results in time units on a similar scale to real runs.
2022-07-04 11:13:07 -07:00
Arseny Kapoulkine 4cd0443913
Update benchmark.yml
Cleaner names
2022-06-24 18:30:26 -07:00
Arseny Kapoulkine 9846a6c7b9
Update benchmark.yml
Remove all alert/comment functionality
2022-06-24 18:26:15 -07:00
Arseny Kapoulkine 224d35bc9e
Update benchmark.yml
Attempt to fix Windows and other builds
2022-06-24 18:16:12 -07:00
Allan N Jeremy 5e405b58b3
Added multi-os runners for benchmark & implemented luau analyze (#542) 2022-06-24 09:46:29 -07:00
Allan N Jeremy da01056022
Added Luau Benchmark Workflows (#530) 2022-06-14 08:48:07 -07:00
Alan Jeffrey d37d0c857b
Prototype: Renamed any/none to unknown/never (#447)
* Renamed any/none to unknown/never
* Pin hackage version
* Update Agda version
2022-04-09 00:07:08 -05:00
Lily Brown 6c923b8802
Prototyping: strings (#390) 2022-03-02 15:26:58 -08:00
Arseny Kapoulkine db3a8a2f0f
Update prototyping.yml (#398)
This limits the scope of prototyping action to PRs to prototyping branch to minimize the GHA cost / latency, as cabal install sometimes takes forever
2022-02-24 17:08:54 -08:00
Arseny Kapoulkine a9bdce6cc0
Rename tests to (#374) 2022-02-18 10:04:38 -08:00
Lily Brown 1ac64af484
Prototyping: Revise CI (#371)
Introduces a test runner with test cases. Also significantly overhauls the GHA configuration.
2022-02-17 17:15:33 -08:00
Lily Brown c8d6dc2758
Revise GHA workflows for prototyping (#367)
Changed the GHA workflows to:
- Not run `build` and `release` workflows for PRs that only affect `prototyping/`
- Run `prototyping` workflow when PRs affect `Analysis/**`, `Ast/**`, or the `luau-ast` source files
2022-02-15 14:24:51 -08:00
Alan Jeffrey db90c7da48
Add a typeToString function to the prototype (#354)
* Added Luau.Type.ToString
2022-02-11 14:38:35 -06:00
Alan Jeffrey 5187e64f88
Implement a prototype interpreter (#353)
* First cut interpreter
2022-02-09 17:14:29 -06:00
Alan Jeffrey 041838a942
Prototyping a small subset of Luau in Agda (#350)
* First cut reading JSON into an Agda representation of Luau syntax
2022-02-08 18:26:58 -06:00
Arseny Kapoulkine 88be067c0b
Switch coverage build to checkout@v2 (#285)
Attempt to fix coverage builds by using checkout@v2 instead of v1 which might fix the detacthed HEAD issue.

On the off chance it doesn't, add extra logging around git specifically.
2021-12-09 17:50:29 -08:00
Arseny Kapoulkine f5ec6df7ba
Update build.yml
Disable continue-on-error for coverage because hiding the error actually makes it difficult to debug now :)
2021-12-02 15:55:06 -08:00
Arseny Kapoulkine dd02420f70
Update build.yml
Enable debug mode in coveralls action to diagnose
2021-11-23 11:44:18 -08:00
Arseny Kapoulkine 5740686124
Ignore errors during upload coverage (#236) 2021-11-23 08:26:28 -08:00
Arseny Kapoulkine 5961261a1c
Add web workflow to build Repl with Emscripten (#222)
This also separates Emscripten build into a new target / source to make
it more decoupled.
2021-11-22 09:59:15 -08:00
Arseny Kapoulkine b7d26b371a
Use -Werror in CI only (#201)
We keep getting compat reports for warnings in various compiler
versions. While we can keep merging PRs to resolve these warnings, it
would be nice if the users of other compilers or compiler versions weren't
blocked on us fixing this.

As such, this change disables Werror by default and only enables it when
requested, which happens in CI in test builds.
2021-11-12 06:56:25 -08:00
Arseny Kapoulkine 7816f2596f
Update config.yml
Rename "help and support" to "questions"
2021-11-08 15:01:57 -08:00
Arseny Kapoulkine 2f7e1a2395
Update CI scripts to ignore .md changes (#100)
This reduces the load on GHA infra
2021-11-03 12:30:30 -07:00
Arseny Kapoulkine 36110e0f29
Update issue template config
Fix discussion link
2021-11-03 09:40:08 -07:00
Arseny Kapoulkine 7f3c033162 Add Coveralls integration
Build and upload lcov-compatible coverage file to coveralls
2021-11-02 15:50:57 -07:00
Arseny Kapoulkine ca965d94ee Add build scaffolding for GHA and issue templates for GH 2021-11-01 14:52:34 -07:00