diff --git a/docs/_pages/compatibility.md b/docs/_pages/compatibility.md index 00d883e..eab1aac 100644 --- a/docs/_pages/compatibility.md +++ b/docs/_pages/compatibility.md @@ -54,7 +54,7 @@ Sandboxing challenges are [covered in the dedicated section](sandbox). | goto statement | ❌ | this complicates the compiler, makes control flow unstructured and doesn't address a significant need | | finalizers for tables | ❌ | no `__gc` support due to sandboxing and performance/complexity | | no more fenv for threads or functions | 😞 | we love this, but it breaks compatibility | -| tables honor the `__len` metamethod | ❌ | performance implications, no strong use cases +| tables honor the `__len` metamethod | 🤷‍♀️ | performance implications, no strong use cases | hex and `\z` escapes in strings | ✔️ | | | support for hexadecimal floats | 🤷‍♀️ | no strong use cases | | order metamethods work for different types | ❌ | no strong use cases and more complicated semantics + compat | @@ -63,7 +63,7 @@ Sandboxing challenges are [covered in the dedicated section](sandbox). | arguments for function called through `xpcall` | ✔️ | | | optional base in `math.log` | ✔️ | | | optional separator in `string.rep` | 🤷‍♀️ | no real use cases | -| new metamethods `__pairs` and `__ipairs` | ❌ | would like to reevaluate iteration design long term | +| new metamethods `__pairs` and `__ipairs` | ❌ | superseded by `__iter` | | frontier patterns | ✔️ | | | `%g` in patterns | ✔️ | | | `\0` in patterns | ✔️ | | @@ -72,7 +72,7 @@ Sandboxing challenges are [covered in the dedicated section](sandbox). Two things that are important to call out here are various new metamethods for tables and yielding in metamethods. In both cases, there are performance implications to supporting this - our implementation is *very* highly tuned for performance, so any changes that affect the core fundamentals of how Lua works have a price. To support yielding in metamethods we'd need to make the core of the VM more involved, since almost every single "interesting" opcode would need to learn how to be resumable - which also complicates future JIT/AOT story. Metamethods in general are important for extensibility, but very challenging to deal with in implementation, so we err on the side of not supporting any new metamethods unless a strong need arises. -For `__pairs`/`__ipairs`, we aren't sure that this is the right design choice - self-iterating tables via `__iter` are very appealing, and if we can resolve some challenges with array iteration order, that would make the language more accessible so we may go that route instead. +For `__pairs`/`__ipairs`, we felt that extending library functions to enable custom containers wasn't the right choice. Instead we revisited iteration design to allow for self-iterating objects via `__iter` metamethod, which results in a cleaner iteration design that also makes it easier to iterate over tables. As such, we have no plans to support `__pairs`/`__ipairs` as all use cases for it can now be solved by `__iter`. Ephemeron tables may be implemented at some point since they do have valid uses and they make weak tables semantically cleaner, however the cleanup mechanism for these is expensive and complicated, and as such this can only be considered after the pending GC rework is complete. diff --git a/docs/_pages/sandbox.md b/docs/_pages/sandbox.md index 409a092..d1d7d11 100644 --- a/docs/_pages/sandbox.md +++ b/docs/_pages/sandbox.md @@ -4,7 +4,7 @@ title: Sandboxing toc: true --- -Luau is safe to embed. Broadly speaking, this means that even in the face of untrusted (and in Roblox case, actively malicious) code, the language and the standard library don't allow any unsafe access to the underlying system, and don't have any bugs that allow escaping out of the sandbox (e.g. to gain native code execution through ROP gadgets et al). Additionally, the VM provides extra features to implement isolation of privileged code from unprivileged code and protect one from the other; this is important if the embedding environment (Roblox) decides to expose some APIs that may not be safe to call from untrusted code, for example because they do provide controlled access to the underlying system or risk PII exposure through fingerprinting etc. +Luau is safe to embed. Broadly speaking, this means that even in the face of untrusted (and in Roblox case, actively malicious) code, the language and the standard library don't allow any unsafe access to the underlying system, and don't have any bugs that allow escaping out of the sandbox (e.g. to gain native code execution through ROP gadgets et al). Additionally, the VM provides extra features to implement isolation of privileged code from unprivileged code and protect one from the other; this is important if the embedding environment decides to expose some APIs that may not be safe to call from untrusted code, for example because they do provide controlled access to the underlying system or risk PII exposure through fingerprinting etc. This safety is achieved through a combination of removing features from the standard library that are unsafe, adding features to the VM that make it possible to implement sandboxing and isolation, and making sure the implementation is safe from memory safety issues using fuzzing. @@ -19,7 +19,7 @@ The following libraries and global functions have been removed as a result: - `io.` library has been removed entirely, as it gives access to files and allows running processes - `package.` library has been removed entirely, as it gives access to files and allows loading native modules - `os.` library has been cleaned up from file and environment access functions (`execute`, `exit`, etc.). The only supported functions in the library are `clock`, `date`, `difftime` and `time`. -- `debug.` library has been removed to a large extent, as it has functions that aren't memory safe and other functions break isolation; the only supported functions are `traceback` ~~and `getinfo` (with reduced functionality)~~. +- `debug.` library has been removed to a large extent, as it has functions that aren't memory safe and other functions break isolation; the only supported functions are `traceback` and `info` (which is similar to `debug.getinfo` but has a slightly different interface). - `dofile` and `loadfile` allowed access to file system and have been removed. To achieve memory safety, access to function bytecode has been removed. Bytecode is hard to validate and using untrusted bytecode may lead to exploits. Thus, `loadstring` doesn't work with bytecode inputs, and `string.dump`/`load` have been removed as they aren't necessary anymore. When embedding Luau, bytecode should be encrypted/signed to prevent MITM attacks as well, as the VM assumes that the bytecode was generated by the Luau compiler (which never produces invalid/unsafe bytecode). @@ -54,7 +54,7 @@ This mechanism is bad for performance, memory safety and isolation: - In Lua 5.1, `__gc` support requires traversing userdata lists redundantly during garbage collection to filter out finalizable objects - In later versions of Lua, userdata that implement `__gc` are split into separate lists; however, finalization prolongs the lifetime of the finalized objects which results in less prompt memory reclamation, and two-step destruction results in extra cache misses for userdata -- `__gc` runs during garbage collection in context of an arbitrary thread which makes the thread identity mechanism described above invalid +- `__gc` runs during garbage collection in context of an arbitrary thread which makes the thread identity mechanism used in Roblox to support trusted Luau code invalid - Objects can be removed from weak tables *after* being finalized, which means that accessing these objects can result in memory safety bugs, unless all exposed userdata methods guard against use-after-gc. - If `__gc` method ever leaks to scripts, they can call it directly on an object and use any method exposed by that object after that. This means that `__gc` and all other exposed methods must support memory safety when called on a destroyed object. diff --git a/docs/_pages/syntax.md b/docs/_pages/syntax.md index 4d39e46..fe825fd 100644 --- a/docs/_pages/syntax.md +++ b/docs/_pages/syntax.md @@ -196,3 +196,26 @@ local sign = if x < 0 then -1 elseif x > 0 then 1 else 0 ``` **Note:** In Luau, the `if-then-else` expression is preferred vs the standard Lua idiom of writing `a and b or c` (which roughly simulates a ternary operator). However, the Lua idiom may return an unexpected result if `b` evaluates to false. The `if-then-else` expression will behave as expected in all situations. + +## Generalized iteration + +Luau uses the standard Lua syntax for iterating through containers, `for vars in values`, but extends the semantics with support for generalized iteration. In Lua, to iterate over a table you need to use an iterator like `next` or a function that returns one like `pairs` or `ipairs`. In Luau, you can simply iterate over a table: + +```lua +for k, v in {1, 4, 9} do + assert(k * k == v) +end +``` + +This works for tables but can also be extended for tables or userdata by implementing `__iter` metamethod that is called before the iteration begins, and should return an iterator function like `next` (or a custom one): + +```lua +local obj = { items = {1, 4, 9} } +setmetatable(obj, { __iter = function(o) return next, o.items end }) + +for k, v in obj do + assert(k * k == v) +end +``` + +The default iteration order for tables is specified to be consecutive for elements `1..#t` and unordered after that, visiting every element; similarly to iteration using `pairs`, modifying the table entries for keys other than the current one results in unspecified behavior. diff --git a/rfcs/STATUS.md b/rfcs/STATUS.md index e3e227a..ef55b5c 100644 --- a/rfcs/STATUS.md +++ b/rfcs/STATUS.md @@ -15,12 +15,6 @@ This document tracks unimplemented RFCs. **Status**: Needs implementation -## Sealed/unsealed typing changes - -[RFC: Only strip optional properties from unsealed tables during subtyping](https://github.com/Roblox/luau/blob/master/rfcs/unsealed-table-subtyping-strips-optional-properties.md) - -**Status**: Implemented but not fully rolled out yet. - ## Safe navigation operator [RFC: Safe navigation postfix operator (?)](https://github.com/Roblox/luau/blob/master/rfcs/syntax-safe-navigation-operator.md) @@ -39,10 +33,10 @@ This document tracks unimplemented RFCs. [RFC: Generalized iteration](https://github.com/Roblox/luau/blob/master/rfcs/generalized-iteration.md) -**Status**: Needs implementation +**Status**: Implemented but not fully rolled out yet. ## Lower Bounds Calculation [RFC: Lower bounds calculation](https://github.com/Roblox/luau/blob/master/rfcs/lower-bounds-calculation.md) -**Status**: Needs implementation +**Status**: Implemented but not fully rolled out yet. diff --git a/rfcs/unsealed-table-subtyping-strips-optional-properties.md b/rfcs/unsealed-table-subtyping-strips-optional-properties.md index deecfdb..d99c1f8 100644 --- a/rfcs/unsealed-table-subtyping-strips-optional-properties.md +++ b/rfcs/unsealed-table-subtyping-strips-optional-properties.md @@ -1,5 +1,7 @@ # Only strip optional properties from unsealed tables during subtyping +**Status**: Implemented + ## Summary Currently subtyping allows optional properties to be stripped from table types during subtyping.