Merge branch 'master' into merge

Analysis/src/TypeVar.cpp

@@ -750,13 +750,15 @@ TypeId SingletonTypes::makeStringMetatable()
     TableTypeVar::Props stringLib = {
         {"byte", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, optionalNumber, optionalNumber}), numberVariadicList})}},
         {"char", {arena->addType(FunctionTypeVar{numberVariadicList, arena->addTypePack({stringType})})}},
-        {"find", {makeFunction(*arena, stringType, {}, {}, {stringType, optionalNumber, optionalBoolean}, {}, {optionalNumber, optionalNumber})}},
+        {"find", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber, optionalBoolean}),
+                     arena->addTypePack(TypePack{{optionalNumber, optionalNumber}, stringVariadicList})})}},
         {"format", {formatFn}}, // FIXME
         {"gmatch", {gmatchFunc}},
         {"gsub", {gsubFunc}},
         {"len", {makeFunction(*arena, stringType, {}, {}, {}, {}, {numberType})}},
         {"lower", {stringToStringType}},
-        {"match", {makeFunction(*arena, stringType, {}, {}, {stringType, optionalNumber}, {}, {optionalString})}},
+        {"match", {arena->addType(FunctionTypeVar{arena->addTypePack({stringType, stringType, optionalNumber}),
+                      arena->addTypePack(TypePackVar{VariadicTypePack{optionalString}})})}},
         {"rep", {makeFunction(*arena, stringType, {}, {}, {numberType}, {}, {stringType})}},
         {"reverse", {stringToStringType}},
         {"sub", {makeFunction(*arena, stringType, {}, {}, {numberType, optionalNumber}, {}, {stringType})}},

docs/_pages/library.md

@@ -488,7 +488,7 @@ function string.char(args: ...number): string
 Returns the string that contains a byte for every input number; all inputs must be integers in `[0..255]` range.
 
 ```
-function string.find(s: string, p: string, init: number?, plain: boolean?): (number?, number?)
+function string.find(s: string, p: string, init: number?, plain: boolean?): (number?, number?, ...string)
 ```
 
 Tries to find an instance of pattern `p` in the string `s`, starting from position `init` (defaults to 1). When `plain` is true, the search is using raw case-insensitive string equality, otherwise `p` should be a [string pattern](https://www.lua.org/manual/5.3/manual.html#6.4.1). If a match is found, returns the position of the match and the length of the match, followed by the pattern captures; otherwise returns `nil`.
@@ -536,7 +536,7 @@ function string.lower(s: string): string
 Returns a string where each byte corresponds to the lower-case ASCII version of the input byte in the source string.
 
 ```
-function string.match(s: string, p: string, init: number?): (number?, number?)
+function string.match(s: string, p: string, init: number?): ...string?
 ```
 
 Tries to find an instance of pattern `p` in the string `s`, starting from position `init` (defaults to 1). `p` should be a [string pattern](https://www.lua.org/manual/5.3/manual.html#6.4.1). If a match is found, returns all pattern captures, or entire matching substring if no captures are present, otherwise returns `nil`.

docs/_pages/performance.md

@@ -185,3 +185,13 @@ While large tables can be a problem for incremental GC in general since currentl
 The incremental garbage collector in Luau runs three phases for each cycle: mark, atomic and sweep. Mark incrementally traverses all live objects, atomic finishes various operations that need to happen without mutator intervention (see previous section), and sweep traverses all objects in the heap, reclaiming memory used by dead objects and performing minor fixup for live objects. While objects allocated during the mark phase are traversed in the same cycle and thus may get reclaimed, objects allocated during the sweep phase are considered live. Because of this, the faster the sweep phase completes, the less garbage will accumulate; and, of course, the less time sweeping takes the less overhead there is from this phase of garbage collection on the process.
 
 Since sweeping traverses the whole heap, we maximize the efficiency of this traversal by allocating garbage-collected objects of the same size in 16 KB pages, and traversing each page at a time, which is otherwise known as a paged sweeper. This ensures good locality of reference as consecutively swept objects are contiugous in memory, and allows us to spend no memory for each object on sweep-related data or allocation metadata, since paged sweeper doesn't need to be able to free objects without knowing which page they are in. Compared to linked list based sweeping that Lua/LuaJIT implement, paged sweeper is 2-3x faster, and saves 16 bytes per object on 64-bit platforms.
+
+## Function inlining and loop unrolling
+
+By default, the bytecode compiler performs a series of optimizations that result in faster execution of the code, but they preserve both execution semantics and debuggability. For example, a function call is compiled as a function call, which may be observable via `debug.traceback`; a loop is compiled as a loop, which may be observable via `lua_getlocal`. To help improve performance in cases where these restrictions can be relaxed, the bytecode compiler implements additional optimizations when optimization level 2 is enabled (which requires using `-O2` switch when using Luau CLI), namely function inlining and loop unrolling.
+
+Only loops with loop bounds known at compile time, such as `for i=1,4 do`, can be unrolled. The loop body must be simple enough for the optimization to be profitable; compiler uses heuristics to estimate the performance benefit and automatically decide if unrolling should be performed.
+
+Only local functions (defined either as `local function foo` or `local foo = function`) can be inlined. The function body must be simple enough for the optimization to be profitable; compiler uses heuristics to estimate the performance benefit and automatically decide if each call to the function should be inlined instead. Additionally recursive invocations of a function can’t be inlined at this time, and inlining is completely disabled for modules that use `getfenv`/`setfenv` functions.
+
+In both cases, in addition to removing the overhead associated with function calls or loop iteration, these optimizations can additionally benefit by enabling additional optimizations, such as constant folding of expressions dependent on loop iteration variable or constant function arguments, or using more efficient instructions for certain expressions when the inputs to these instructions are constants.

rfcs/STATUS.md

@@ -26,3 +26,9 @@ This document tracks unimplemented RFCs.
 [RFC: Lower bounds calculation](https://github.com/Roblox/luau/blob/master/rfcs/lower-bounds-calculation.md)
 
 **Status**: Implemented but not fully rolled out yet.
+
+## never and unknown types
+
+[RFC: never and unknown types](https://github.com/Roblox/luau/blob/master/rfcs/never-and-unknown-types.md)
+
+**Status**: Needs implementation

rfcs/never-and-unknown-types.md

@@ -0,0 +1,144 @@
+# never and unknown types
+
+## Summary
+
+Add `unknown` and `never` types that are inhabited by everything and nothing respectively.
+
+## Motivation
+
+There are lots of cases in local type inference, semantic subtyping,
+and type normalization, where it would be useful to have top and
+bottom types. Currently, `any` is filling that role, but it has
+special "switch off the type system" superpowers.
+
+Any use of `unknown` must be narrowed by type refinements unless another `unknown` or `any` is expected. For
+example a function which can return any value is:
+
+```lua
+  function anything() : unknown ... end
+```
+
+and can be used as:
+
+```lua
+  local x = anything()
+  if type(x) == "number" then
+    print(x + 1)
+  end
+```
+
+The type of this function cannot be given concisely in current
+Luau. The nearest equivalent is `any`, but this switches off the type system, for example
+if the type of `anything` is `() -> any` then the following code typechecks:
+
+```lua
+  local x = anything()
+  print(x + 1)
+```
+
+This is fine in nonstrict mode, but strict mode should flag this as an error.
+
+The `never` type comes up whenever type inference infers incompatible types for a variable, for example
+
+```lua
+  function oops(x)
+    print("hi " .. x)  -- constrains x must be a string
+    print(math.abs(x)) -- constrains x must be a number
+  end
+```
+
+The most general type of `x` is `string & number`, so this code gives
+a type error, but we still need to provide a type for `oops`. With a
+`never` type, we can infer the type `oops : (never) -> ()`.
+
+or when exhaustive type casing is achieved:
+
+```lua
+  function f(x: string | number)
+    if type(x) == "string" then
+      -- x : string
+    elseif type(x) == "number" then
+      -- x : number
+    else
+      -- x : never
+    end
+  end
+```
+
+or even when the type casing is simply nonsensical:
+
+```lua
+  function f(x: string | number)
+    if type(x) == "string" and type(x) == "number" then
+      -- x : string & number which is never
+    end
+  end
+```
+
+The `never` type is also useful in cases such as tagged unions where
+some of the cases are impossible. For example:
+
+```lua
+  type Result<T, E> = { err: false, val: T } | { err: true, err: E }
+```
+
+For code which we know is successful, we would like to be able to
+indicate that the error case is impossible. With a `never` type, we
+can do this with `Result<T, never>`. Similarly, code which cannot succeed
+has type `Result<never, E>`.
+
+These types can _almost_ be defined in current Luau, but only quite verbosely:
+
+```lua
+  type never = number & string
+  type unknown = nil | number | boolean | string | {} | (...never) -> (...unknown)
+```
+
+But even for `unknown` it is impossible to include every single data types, e.g. every root class.
+
+Providing `never` and `unknown` as built-in types makes the code for
+type inference simpler, for example we have a way to present a union
+type with no options (as `never`). Otherwise we have to contend with ad hoc
+corner cases.
+
+## Design
+
+Add:
+
+* a type `never`, inhabited by nothing, and
+* a type `unknown`, inhabited by everything.
+
+And under success types (nonstrict mode), `unknown` is exactly equivalent to `any` because `unknown`
+encompasses everything as does `any`.
+
+The interesting thing is that `() -> (never, string)` is equivalent to `() -> never` because all
+values in a pack must be inhabitable in order for the pack itself to also be inhabitable. In fact,
+the type `() -> never` is not completely accurate, it should be `() -> (never, ...never)` to avoid
+cascading type errors. Ditto for when an expression list `f(), g()` where the resulting type pack is
+`(never, string, number)` is still the same as `(never, ...never)`.
+
+```lua
+  function f(): never error() end
+  function g(): string return "" end
+
+  -- no cascading type error where count mismatches, because the expression list f(), g()
+  -- was made to return (never, ...never) due to the presence of a never type in the pack
+  local x, y, z = f(), g()
+  -- x : never
+  -- y : never
+  -- z : never
+```
+
+## Drawbacks
+
+Another bit of complexity budget spent.
+
+These types will be visible to creators, so yay bikeshedding!
+
+Replacing `any` with `unknown` is a breaking change: code in strict mode may now produce errors.
+
+## Alternatives
+
+Stick with the current use of `any` for these cases.
+
+Make `never` and `unknown` type aliases rather than built-ins.