Skip to content

Commit

Permalink
Format the rest of our markdown
Browse files Browse the repository at this point in the history
Summary:
Stacked the next diff on top and ran

```
arc lint --take PRETTIERMARKDOWN --paths-cmd 'find . -type f -name "*.md" -print' -a
```

Reviewed By: zertosh

Differential Revision: D53590286

fbshipit-source-id: 52cad14893ef6894e36646a1de22f1da55e6f58d
  • Loading branch information
JakobDegen authored and facebook-github-bot committed Feb 8, 2024
1 parent c04d1bb commit df4db31
Show file tree
Hide file tree
Showing 13 changed files with 700 additions and 391 deletions.
277 changes: 153 additions & 124 deletions CHANGELOG.md

Large diffs are not rendered by default.

13 changes: 7 additions & 6 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
# Contributing to Starlark

We want to make contributing to this project as easy and transparent as possible.
We want to make contributing to this project as easy and transparent as
possible.

## Our Development Process

Starlark is currently developed in Facebook's internal repositories and then exported
out to GitHub by a Facebook team member; however, we invite you to submit pull
requests as described below.
Starlark is currently developed in Facebook's internal repositories and then
exported out to GitHub by a Facebook team member; however, we invite you to
submit pull requests as described below.

## Pull Requests

Expand Down Expand Up @@ -41,5 +42,5 @@ Follow the automatic `rust fmt` configuration.

## License

By contributing to Starlark, you agree that your contributions will be
licensed under the [LICENSE](LICENSE) file in the root directory of this source tree.
By contributing to Starlark, you agree that your contributions will be licensed
under the [LICENSE](LICENSE) file in the root directory of this source tree.
136 changes: 100 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,25 @@
[![docs.rs availability](https://img.shields.io/docsrs/starlark?label=docs.rs)](https://docs.rs/starlark/)
[![Build status](https://img.shields.io/github/actions/workflow/status/facebookexperimental/starlark-rust/ci.yml?branch=main)](https://github.com/facebookexperimental/starlark-rust/actions)

There are several copies of this repo on GitHub, [facebookexperimental/starlark-rust](https://github.com/facebookexperimental/starlark-rust) is the canonical one.

This project provides a Rust implementation of the [Starlark language](https://github.com/bazelbuild/starlark/blob/master/spec.md). Starlark (formerly codenamed Skylark) is a deterministic language inspired by Python3, used for configuration in the build systems [Bazel](https://bazel.build), [Buck](https://buck.build) and [Buck2](https://buck2.build), of which Buck2 depends on this library. This project was originally developed [in this repo](https://github.com/google/starlark-rust), which contains a more extensive history.

There are at least three implementations of Starlark, [one in Java](https://github.com/bazelbuild/starlark), [one in Go](https://github.com/google/starlark-go), and this one in Rust. We mostly follow the Starlark standard. If you are interested in trying out Rust Starlark, you can clone this repo and run:
There are several copies of this repo on GitHub,
[facebookexperimental/starlark-rust](https://github.com/facebookexperimental/starlark-rust)
is the canonical one.

This project provides a Rust implementation of the
[Starlark language](https://github.com/bazelbuild/starlark/blob/master/spec.md).
Starlark (formerly codenamed Skylark) is a deterministic language inspired by
Python3, used for configuration in the build systems
[Bazel](https://bazel.build), [Buck](https://buck.build) and
[Buck2](https://buck2.build), of which Buck2 depends on this library. This
project was originally developed
[in this repo](https://github.com/google/starlark-rust), which contains a more
extensive history.

There are at least three implementations of Starlark,
[one in Java](https://github.com/bazelbuild/starlark),
[one in Go](https://github.com/google/starlark-go), and this one in Rust. We
mostly follow the Starlark standard. If you are interested in trying out Rust
Starlark, you can clone this repo and run:

```shell
$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
Expand All @@ -19,60 +33,110 @@ $> 1+2
3
```

This project was started by [Damien Martin-Guillerez](https://github.com/damienmg). Version 0.4.0 of this library changed ownership [from Google](https://github.com/google/starlark-rust) to Facebook.
This project was started by
[Damien Martin-Guillerez](https://github.com/damienmg). Version 0.4.0 of this
library changed ownership [from Google](https://github.com/google/starlark-rust)
to Facebook.

## Learn More

Read [this blog post](https://developers.facebook.com/blog/post/2021/04/08/rust-starlark-library/) for an overview of the library, the reasons behind Starlark, and how it might fit in to your project. There is also a [2 minute introductory video](https://www.youtube.com/watch?v=3kHER3KIPj4).
Read
[this blog post](https://developers.facebook.com/blog/post/2021/04/08/rust-starlark-library/)
for an overview of the library, the reasons behind Starlark, and how it might
fit in to your project. There is also a
[2 minute introductory video](https://www.youtube.com/watch?v=3kHER3KIPj4).

## Features

This project features:

* Easy interoperability between Rust types and Starlark.
* Rust-friendly types, so frozen values are `Send`/`Sync`, while non-frozen values aren't.
* [Garbage collected](docs/gc.md) values allocated on [a heap](docs/heaps.md).
* Optional runtime-checked [types](docs/types.md).
* A linter, to detect code issues in Starlark.
* IDE integration in the form of [LSP](https://microsoft.github.io/language-server-protocol/).
* Extensive testing, including [fuzz testing](https://github.com/google/oss-fuzz/tree/master/projects/starlark-rust).
* [DAP](https://microsoft.github.io/debug-adapter-protocol/) support.
- Easy interoperability between Rust types and Starlark.
- Rust-friendly types, so frozen values are `Send`/`Sync`, while non-frozen
values aren't.
- [Garbage collected](docs/gc.md) values allocated on [a heap](docs/heaps.md).
- Optional runtime-checked [types](docs/types.md).
- A linter, to detect code issues in Starlark.
- IDE integration in the form of
[LSP](https://microsoft.github.io/language-server-protocol/).
- Extensive testing, including
[fuzz testing](https://github.com/google/oss-fuzz/tree/master/projects/starlark-rust).
- [DAP](https://microsoft.github.io/debug-adapter-protocol/) support.

This project also has three non-goals:

* We do _not_ aim for API stability between releases, preferring to iterate quickly and refine the API as much as possible. But we do [follow SemVer](https://doc.rust-lang.org/cargo/reference/semver.html).
* We do _not_ aim for minimal dependencies, preferring to keep one package with lots of power. But if some dependencies prove tricky, we might add feature flags.
- We do _not_ aim for API stability between releases, preferring to iterate
quickly and refine the API as much as possible. But we do
[follow SemVer](https://doc.rust-lang.org/cargo/reference/semver.html).
- We do _not_ aim for minimal dependencies, preferring to keep one package with
lots of power. But if some dependencies prove tricky, we might add feature
flags.

## Components

There are six components:

* `starlark_derive`, a proc-macro crate that defines the necessary macros for Starlark. This library is a dependency of `starlark` the library, which reexports all the relevant pieces, and should not be used directly.
* `starlark_map`, a library with memory-efficient ordered/unordered maps/sets and various other data structures useful in Starlark.
* `starlark_syntax`, a library with the AST of Starlark and parsing functions. Only use if you want to manipulate the AST directly.
* `starlark` the main library, with evaluator, standard library, debugger support and lots of other pieces. Projects wishing to embed Starlark in their environment (with additional types, library functions and features) will make use of this library. This library reexports the relevant pieces of `starlark_derive`, `starlark_map` and most of `starlark_syntax`.
* `starlark_lsp`, a library providing an [LSP](https://microsoft.github.io/language-server-protocol/).
* `starlark_bin` the binary, which provides interactive evaluation, IDE features and linter, exposed through a command line. Useful if you want to use vanilla Starlark (but if you do, consider Python3 instead) or as a test-bed for experimenting. Most projects will end up implementing some of this functionality themselves over the `starlark` and `starlark_lsp` libraries, incorporating their specific extra types etc.

In particular the `starlark_bin` binary _can_ be effectively used as a linter. But for the REPL, evaluator and IDE features the `starlark_bin` binary is only aware of standard Starlark. Most Starlark embeddings supply extra functions and data types to work with domain-specific concerns, and the lack of these bindings will cause the REPL/evaluator to fail if they are used, and will give a subpar IDE experience. In most cases you should write your own binary depending on the `starlark` library, integrating your domain-specific pieces, and then using the bundled LSP functions to produce your own IDE/REPL/evaluator on top of those. You should still be able to use the [VS Code extension](vscode/README.md).
- `starlark_derive`, a proc-macro crate that defines the necessary macros for
Starlark. This library is a dependency of `starlark` the library, which
reexports all the relevant pieces, and should not be used directly.
- `starlark_map`, a library with memory-efficient ordered/unordered maps/sets
and various other data structures useful in Starlark.
- `starlark_syntax`, a library with the AST of Starlark and parsing functions.
Only use if you want to manipulate the AST directly.
- `starlark` the main library, with evaluator, standard library, debugger
support and lots of other pieces. Projects wishing to embed Starlark in their
environment (with additional types, library functions and features) will make
use of this library. This library reexports the relevant pieces of
`starlark_derive`, `starlark_map` and most of `starlark_syntax`.
- `starlark_lsp`, a library providing an
[LSP](https://microsoft.github.io/language-server-protocol/).
- `starlark_bin` the binary, which provides interactive evaluation, IDE features
and linter, exposed through a command line. Useful if you want to use vanilla
Starlark (but if you do, consider Python3 instead) or as a test-bed for
experimenting. Most projects will end up implementing some of this
functionality themselves over the `starlark` and `starlark_lsp` libraries,
incorporating their specific extra types etc.

In particular the `starlark_bin` binary _can_ be effectively used as a linter.
But for the REPL, evaluator and IDE features the `starlark_bin` binary is only
aware of standard Starlark. Most Starlark embeddings supply extra functions and
data types to work with domain-specific concerns, and the lack of these bindings
will cause the REPL/evaluator to fail if they are used, and will give a subpar
IDE experience. In most cases you should write your own binary depending on the
`starlark` library, integrating your domain-specific pieces, and then using the
bundled LSP functions to produce your own IDE/REPL/evaluator on top of those.
You should still be able to use the [VS Code extension](vscode/README.md).

## Compatibility

In this section we outline where we don't comply with the [Starlark spec](https://github.com/bazelbuild/starlark/blob/master/spec.md).
In this section we outline where we don't comply with the
[Starlark spec](https://github.com/bazelbuild/starlark/blob/master/spec.md).

* We have plenty of extensions, e.g. type annotations, recursion, top-level `for`.
* We don't yet support later additions to Starlark, such as [bytes](https://github.com/facebookexperimental/starlark-rust/issues/4).
* In some cases creating circular data structures may lead to stack overflows.
- We have plenty of extensions, e.g. type annotations, recursion, top-level
`for`.
- We don't yet support later additions to Starlark, such as
[bytes](https://github.com/facebookexperimental/starlark-rust/issues/4).
- In some cases creating circular data structures may lead to stack overflows.

## Making a release

1. Check the [GitHub Actions](https://github.com/facebookexperimental/starlark-rust/actions) are green.
2. Update `CHANGELOG.md` with the changes since the last release. [This link](https://github.com/facebookexperimental/starlark-rust/compare/v0.4.0...main) can help (update to compare against the last release).
3. Update the version numbers of the two `Cargo.toml` files. Bump them by 0.0.1 if there are no incompatible changes, or 0.1.0 if there are. Bump the dependency in `starlark` to point at the latest `starlark_derive` version.
4. Copy the files `CHANGELOG.md`, `LICENSE` and `README.md` into each subdirectory.
5. Run `cargo publish --allow-dirty --dry-run`, then without the `--dry-run`, in each of the component directories in the [order above](#components).
6. Create a [GitHub release](https://github.com/facebookexperimental/starlark-rust/releases/new) with `v0.X.Y`, using the `starlark` version as the name.
1. Check the
[GitHub Actions](https://github.com/facebookexperimental/starlark-rust/actions)
are green.
2. Update `CHANGELOG.md` with the changes since the last release.
[This link](https://github.com/facebookexperimental/starlark-rust/compare/v0.4.0...main)
can help (update to compare against the last release).
3. Update the version numbers of the two `Cargo.toml` files. Bump them by 0.0.1
if there are no incompatible changes, or 0.1.0 if there are. Bump the
dependency in `starlark` to point at the latest `starlark_derive` version.
4. Copy the files `CHANGELOG.md`, `LICENSE` and `README.md` into each
subdirectory.
5. Run `cargo publish --allow-dirty --dry-run`, then without the `--dry-run`, in
each of the component directories in the [order above](#components).
6. Create a
[GitHub release](https://github.com/facebookexperimental/starlark-rust/releases/new)
with `v0.X.Y`, using the `starlark` version as the name.

## License

Starlark Rust is Apache License, Version 2.0 licensed, as found in the [LICENSE](LICENSE) file.
Starlark Rust is Apache License, Version 2.0 licensed, as found in the
[LICENSE](LICENSE) file.
45 changes: 23 additions & 22 deletions allocative/README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,42 @@
# Allocative: memory profiler for Rust

This crate implements a lightweight memory profiler which allows
object traversal and memory size introspection.
This crate implements a lightweight memory profiler which allows object
traversal and memory size introspection.

## Usage

`Allocative` trait (typically implemented with proc-macro) is introspectable:
`Allocative` values can be traversed and their size and sizes of referenced objects
can be collected.
`Allocative` values can be traversed and their size and sizes of referenced
objects can be collected.

This crate provides a few utilities to work with such objects,
the main of such utilities is flame graph builder which produces flame graph
(see the crate documentation) like this:
This crate provides a few utilities to work with such objects, the main of such
utilities is flame graph builder which produces flame graph (see the crate
documentation) like this:

![sample-flamegraph.png](sample-flamegraph.png)

## How it is different from other call-stack malloc profilers like jemalloc heap profiler

Allocative is not a substitute for call stack malloc profiler,
it provides a different view of memory usage.
Allocative is not a substitute for call stack malloc profiler, it provides a
different view of memory usage.

Here are some differences between allocative and call-stack malloc profiler:

* Allocative requires implementation of `Allocative` trait for each type
which needs to be measured, and some setup in the program to enable it is needed
* Allocative flamegraph shows object by object tree, not by call stack
* Allocative shows gaps in allocated memory,
e.g. spare capacity of collections or too large padding in structs or enums
* Allocative allows profiling of non-malloc allocations
(for example, allocations within [bumpalo](https://github.com/fitzgen/bumpalo) bumps)
* Allocative allows profiling of memory for subset of the process data
(for example, measure the size of RPC response before serialization)
- Allocative requires implementation of `Allocative` trait for each type which
needs to be measured, and some setup in the program to enable it is needed
- Allocative flamegraph shows object by object tree, not by call stack
- Allocative shows gaps in allocated memory, e.g. spare capacity of collections
or too large padding in structs or enums
- Allocative allows profiling of non-malloc allocations (for example,
allocations within [bumpalo](https://github.com/fitzgen/bumpalo) bumps)
- Allocative allows profiling of memory for subset of the process data (for
example, measure the size of RPC response before serialization)

## Runtime overhead

When allocative is used, binary size is slightly increased due to implementations
of [`Allocative`] trait, but it has no runtime/memory overhead when it is enabled but not used.
When allocative is used, binary size is slightly increased due to
implementations of [`Allocative`] trait, but it has no runtime/memory overhead
when it is enabled but not used.

## Source code

Expand All @@ -45,5 +46,5 @@ is synchronized to GitHub. The main copy is

## License

Allocative is both MIT and Apache License, Version 2.0 licensed,
as found in the [LICENSE-MIT](LICENSE-MIT) and [LICENSE-APACHE](LICENSE-APACHE) files.
Allocative is both MIT and Apache License, Version 2.0 licensed, as found in the
[LICENSE-MIT](LICENSE-MIT) and [LICENSE-APACHE](LICENSE-APACHE) files.
Loading

0 comments on commit df4db31

Please sign in to comment.