Skip to content

Commit

Permalink
Merge pull request #646 from obsidiansystems/v0.5
Browse files Browse the repository at this point in the history
v0.5
  • Loading branch information
3noch authored Feb 7, 2020
2 parents 62131ac + f9c707f commit 57fd855
Show file tree
Hide file tree
Showing 34 changed files with 1,078 additions and 605 deletions.
9 changes: 9 additions & 0 deletions ChangeLog.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,15 @@

This project's release branch is `master`. This log is written from the perspective of the release branch: when changes hit `master`, they are considered released, and the date should reflect that release.

## v0.5.0.0 - 2020-02-07

* Add `Obelisk.Route.(?/)`, a convenience function for constructing routes nested in `Maybe`. ([#457](https://github.com/obsidiansystems/obelisk/pull/457))
* Add local unpacked packages to the `ob run`, `ob watch`, and `ob repl` sessions. Any `.cabal` or hpack package inside the current obelisk project will be loaded into the session. For `ob run`/`ob watch` this means the session will automatically reload when you save a source file in any of those packages. For `ob repl` it means that `:r` will reload changes to any of those packages. There are some edge cases where this integration is still rough. Report any issues you encounter. ([#489](https://github.com/obsidiansystems/obelisk/pull/489))
* Add `ob hoogle` command to start a local [Hoogle](https://hoogle.haskell.org/) server for the project. ([#628](https://github.com/obsidiansystems/obelisk/pull/628))
* `ob thunk pack` will now attempt to automatically detect if the thunk is a private or public repo. To avoid this detection, specify `--private` or `--public` manually. ([#607](https://github.com/obsidiansystems/obelisk/pull/607))
* Fix a bug in the plain git thunk loader for thunks marked as 'private' when the revision is not in the default branch. ([#648](https://github.com/obsidiansystems/obelisk/pull/648))
* Improve handling of runtime nix dependencies. This may fix some issues encountered particularly by users on systems other than NixOS.

## v0.4.0.0 - 2020-01-10

* Bump reflex-platform which, notably, bumps nixpkgs to 19.09. ([#585](https://github.com/obsidiansystems/obelisk/pull/585))
Expand Down
30 changes: 30 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
Copyright (c) 2020 Obsidian Systems

All rights reserved.

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

* Redistributions of source code must retain the above copyright
notice, this list of conditions and the following disclaimer.

* Redistributions in binary form must reproduce the above
copyright notice, this list of conditions and the following
disclaimer in the documentation and/or other materials provided
with the distribution.

* Neither the name of Elliot Cameron nor the names of other
contributors may be used to endorse or promote products derived
from this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
37 changes: 26 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
# Obelisk

[![Haskell Programming Language](https://img.shields.io/badge/language-Haskell-blue.svg)](http://www.haskell.org)
[![BSD3 License](http://img.shields.io/badge/license-BSD3-brightgreen.svg)](https://tldrlegal.com/license/bsd-3-clause-license-%28revised%29)


Obelisk provides an easy way to develop and deploy your [Reflex](https://github.com/reflex-frp/reflex) project as web apps and as mobile apps.

- [Installing Obelisk](#installing-obelisk)
- [Developing an Obelisk project](#developing-an-obelisk-project)
- [Hoogle](#hoogle)
- [Adding Packages](#adding-packages)
- [Adding Package Overrides](#adding-package-overrides)
- [Running over https](#running-over-https)
Expand All @@ -23,14 +26,15 @@ Obelisk provides an easy way to develop and deploy your [Reflex](https://github.
1. Set up nix caches
1. If you are running NixOS, add this to `/etc/nixos/configuration.nix`:
```
nix.binaryCaches = [ "https://cache.nixos.org/" "https://nixcache.reflex-frp.org" ];
nix.binaryCaches = [ "https://nixcache.reflex-frp.org" ];
nix.binaryCachePublicKeys = [ "ryantrinkle.com-1:JJiAKaRv9mWgpVAz8dwewnZe0AzzEAzPkagE9SP5NWI=" ];
```
and rebuild your NixOS configuration (e.g. `sudo nixos-rebuild switch`).
1. If you are using another operating system or linux distribution, ensure that these lines are present in your Nix configuration file (`/etc/nix/nix.conf` on most systems; [see full list](https://nixos.org/nix/manual/#sec-conf-file)):
```
substituters = https://cache.nixos.org https://nixcache.reflex-frp.org
trusted-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= ryantrinkle.com-1:JJiAKaRv9mWgpVAz8dwewnZe0AzzEAzPkagE9SP5NWI=
binary-caches = https://cache.nixos.org https://nixcache.reflex-frp.org
binary-cache-public-keys = cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY= ryantrinkle.com-1:JJiAKaRv9mWgpVAz8dwewnZe0AzzEAzPkagE9SP5NWI=
binary-caches-parallel-connections = 40
```
* other Linux: enable sandboxing (see these [issue172](https://github.com/obsidiansystems/obelisk/issues/172#issuecomment-411507818) or [issue6](https://github.com/obsidiansystems/obelisk/issues/6) if you run into build problems)
```
Expand Down Expand Up @@ -111,12 +115,6 @@ Firefox will not be able to properly run the development website due to [issue 4

Every time you change the Haskell source files in frontend, common or backend, `ob run` will automatically recompile the modified files and reload the server. Furthermore, it will display on screen compilation errors and warnings if any.

### Hoogle

To enter a nix-shell from which you can run the hoogle command-line client or a hoogle server for your project:

`nix-shell -A shells.ghc --arg withHoogle true`

### Adding packages

In order to add package dependencies, declare them under the build-depends field in the appropriate cabal files (backend, common, and frontend each have their own). The corresponding Nix packages will automatically be selected when building.
Expand Down Expand Up @@ -150,6 +148,23 @@ project ./. ({ pkgs, ... }: {

For further information see [the Haskell section](https://nixos.org/nixpkgs/manual/#users-guide-to-the-haskell-infrastructure) of nixpkgs Contributors Guide.

### Adding extra local packages

If the standard packages (`frontend`, `backend`, and `common`) are not
enough, to add more local Haskell packages, define them with the
`packages` parameter. The sources of these packages will be
automatically reloaded by `ob run`.

```nix
# ...
project ./. ({ pkgs, ... }: {
# ...
packages = {
another = ./another;
};
# ...
```

### Running over https

To run your app locally over https, update the protocol in `config/common/route` to `https`, and then use `ob run` as normal.
Expand Down Expand Up @@ -181,7 +196,7 @@ In this section we will demonstrate how to deploy your Obelisk app to an Amazon

First create a new EC2 instance:

1. Launch a NixOS 18.09 EC2 instance (we recommend [this AMI](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#LaunchInstanceWizard:ami=ami-009c9c3f1af480ff3))
1. Launch a NixOS 19.09 EC2 instance (we recommend [this AMI](https://console.aws.amazon.com/ec2/v2/home?region=us-east-1#LaunchInstanceWizard:ami=ami-00a8eeaf232a74f84))
1. In the instance configuration wizard ensure that your instance has at least 1GB RAM and 10GB disk space.
1. When prompted save your AWS private key (`~/myaws.pem`) somewhere safe. We'll need it later during deployment.
1. Go to "Security Groups", select your instance's security group and under "Inbound" tab add a new rule for HTTP port 80 and 443.
Expand Down
113 changes: 61 additions & 52 deletions all-builds.nix
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@
iosSdkVersion = "10.2";
}
, local-self ? import ./. self-args
, supportedSystems ? [ "x86_64-linux" "x86_64-darwin" ]
, supportedSystems ? [
"x86_64-linux"
"x86_64-darwin"
]
}:

let
Expand Down Expand Up @@ -35,59 +38,65 @@ let
(name: { inherit name; value = pkgsSet.${name}; })
pnames);

concatDepends = let
extractDeps = x: (x.override {
mkDerivation = drv: {
out = builtins.concatLists [
(drv.buildDepends or [])
(drv.libraryHaskellDepends or [])
(drv.executableHaskellDepends or [])
];
};
}).out;
in pkgAttrs: builtins.concatLists (map extractDeps (builtins.attrValues pkgAttrs));
collect = v:
if lib.isDerivation v then [v]
else if lib.isAttrs v then lib.concatMap collect (builtins.attrValues v)
else if lib.isList v then lib.concatMap collect v
else [];

perPlatform = lib.genAttrs cacheBuildSystems (system: let
obelisk = import ./. (self-args // { inherit system; });
reflex-platform = obelisk.reflex-platform;
ghc = pnameToAttrs
obelisk.haskellPackageSets.ghc
obeliskPackagesBackend;
ghcjs = pnameToAttrs
obelisk.haskellPackageSets.ghcjs
obeliskPackagesCommon;
cachePackages = builtins.concatLists [
(builtins.attrValues ghc)
(builtins.attrValues ghcjs)
(concatDepends ghc)
(concatDepends ghcjs)
(lib.optional reflex-platform.androidSupport androidSkeleton)
(lib.optional reflex-platform.iosSupport iosSkeleton)
[ command serverSkeletonExe serverSkeletonShell ]
];
command = obelisk.command;
skeleton = import ./skeleton { inherit obelisk; };
serverSkeletonExe = skeleton.exe;
# TODO fix nixpkgs so it doesn't try to run the result of haskell shells as setup hooks.
serverSkeletonShell = local-self.nixpkgs.runCommand "shell-safe-for-dep" {} ''
touch "$out"
echo "return" >> "$out"
cat "${skeleton.shells.ghc}" >> "$out"
'';
androidSkeleton = (import ./skeleton { inherit obelisk; }).android.frontend;
iosSkeleton = (import ./skeleton { inherit obelisk; }).ios.frontend;
in {
inherit
command
ghc ghcjs
serverSkeletonExe
serverSkeletonShell
;
cache = reflex-platform.pinBuildInputs "obelisk-${system}" cachePackages;
} // lib.optionalAttrs reflex-platform.androidSupport {
inherit androidSkeleton;
} // lib.optionalAttrs reflex-platform.iosSupport {
inherit iosSkeleton;
reflex-platform = import ./dep/reflex-platform { inherit system; };

mkPerProfiling = profiling: let
obelisk = import ./. (self-args // { inherit system profiling; });
ghc = pnameToAttrs
obelisk.haskellPackageSets.ghc
obeliskPackagesBackend;
ghcjs = pnameToAttrs
obelisk.haskellPackageSets.ghcjs
obeliskPackagesCommon;
command = obelisk.command;
skeleton = import ./skeleton { inherit obelisk; };
serverSkeletonExe = skeleton.exe;
# TODO fix nixpkgs so it doesn't try to run the result of haskell shells as setup hooks.
serverSkeletonShell = local-self.nixpkgs.runCommand "shell-safe-for-dep" {} ''
touch "$out"
echo "return" >> "$out"
cat "${skeleton.shells.ghc}" >> "$out"
'';
androidSkeleton = (import ./skeleton { inherit obelisk; }).android.frontend;
iosSkeleton = (import ./skeleton { inherit obelisk; }).ios.frontend;
nameSuffix = if profiling then "profiled" else "unprofiled";
packages = {
inherit
command
serverSkeletonShell
ghc
;
} // lib.optionalAttrs (!profiling) {
inherit
ghcjs
serverSkeletonExe
;
} // lib.optionalAttrs reflex-platform.androidSupport {
inherit androidSkeleton;
} // lib.optionalAttrs reflex-platform.iosSupport {
inherit iosSkeleton;
};
in packages // {
cache = reflex-platform.pinBuildInputs
"obelisk-${system}-${nameSuffix}"
(collect packages);
};

perProfiling = {
profiled = mkPerProfiling true;
unprofiled = mkPerProfiling false;
};
in perProfiling // {
cache = reflex-platform.pinBuildInputs
"obelisk-${system}"
(map (p: p.cache) (builtins.attrValues perProfiling));
});

metaCache = local-self.reflex-platform.pinBuildInputs
Expand Down
66 changes: 52 additions & 14 deletions all-tests.nix
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
supportedSystems ? [ builtins.currentSystem ]
}:
let
let
nginxRoot = "/run/nginx";
obelisk = import ./default.nix {};
# Get NixOS a pre-release 20.03 that contains the python based tests and recursive nix
Expand All @@ -21,7 +21,7 @@ in
enable = true;
};
environment.systemPackages = [
pkgs.git
pkgs.git
];
users.users.root.openssh.authorizedKeys.keys = [
snakeOilPublicKey
Expand All @@ -38,14 +38,20 @@ in
obelisk.command
obelisk.shell
obelisk-everywhere
pkgs.git
pkgs.git
];
};
};

testScript =
let
privateKeyFile = pkgs.writeText "id_rsa" ''${snakeOilPrivateKey}'';
thunkableSample = pkgs.writeText "default.nix" ''
let pkgs = import <nixpkgs> {}; in pkgs.git
'';
invalidThunkableSample = pkgs.writeText "default.nix" ''
let pkgs = import <nixpkgs> {}; in pkgtypo.git
'';
sshConfigFile = pkgs.writeText "ssh_config" ''
Host *
StrictHostKeyChecking no
Expand All @@ -58,10 +64,10 @@ in
in ''
start_all()
githost.wait_for_open_port("22")
with subtest("test obelisk is installed"):
client.succeed("ob --help")
with subtest("test the client can access the server via ssh"):
client.succeed("mkdir -p ~/.ssh/")
client.succeed(
Expand All @@ -75,33 +81,65 @@ in
"cp ${sshConfigFile} ~/.ssh/config"
)
client.wait_until_succeeds("ssh githost true")
with subtest("test a remote bare repo can be started"):
githost.succeed("mkdir -p ~/myorg/myapp.git")
githost.succeed("cd ~/myorg/myapp.git && git init --bare")
with subtest("test a git project can be configured with a remote using ssh"):
client.succeed("mkdir -p ~/code/myapp")
client.succeed("cd ~/code/myapp && git init")
client.succeed("cd ~/code/myapp && touch README")
client.succeed(
"cp ${thunkableSample} ~/code/myapp/default.nix"
)
client.succeed("cd ~/code/myapp && git add .")
client.succeed('git config --global user.email "[email protected]"')
client.succeed('git config --global user.name "Your Name"')
client.succeed('cd ~/code/myapp && git commit -m "Initial"')
client.succeed(
"cd ~/code/myapp && git remote add origin root@githost:/root/myorg/myapp.git"
)
with subtest("test pushing code to the remote"):
client.succeed("cd ~/code/myapp && git push -u origin master")
client.succeed("cd ~/code/myapp && git status")
with subtest("test obelisk can pack"):
client.succeed("ob thunk pack ~/code/myapp")
client.succeed("ob -v thunk pack ~/code/myapp")
client.succeed("grep -qF 'git' ~/code/myapp/default.nix")
client.succeed("grep -qF 'myorg' ~/code/myapp/git.json")
with subtest("test obelisk can detect private repos"):
client.succeed("ob -v thunk unpack ~/code/myapp")
with subtest("test obelisk can set the public / private flag"):
client.succeed("ob -v thunk pack ~/code/myapp --private")
client.fail("""grep -qF '"private": practice' ~/code/myapp/git.json""")
client.succeed("""grep -qF '"private": true' ~/code/myapp/git.json""")
client.succeed("nix-build ~/code/myapp")
client.succeed("ob -v thunk unpack ~/code/myapp")
client.succeed("ob -v thunk pack ~/code/myapp --public")
client.succeed("""grep -qF '"private": false' ~/code/myapp/git.json""")
'';
client.succeed("nix-build ~/code/myapp")
client.succeed("ob -v thunk unpack ~/code/myapp")
with subtest("test building an invalid thunk fails"):
client.succeed("cd ~/code/myapp && git checkout -b bad")
client.succeed(
"cp ${invalidThunkableSample} ~/code/myapp/default.nix"
)
client.succeed("cd ~/code/myapp && git add .")
client.succeed('git config --global user.email "[email protected]"')
client.succeed('git config --global user.name "Your Name"')
client.succeed('cd ~/code/myapp && git commit -m "Bad commit"')
client.succeed("cd ~/code/myapp && git push -u origin bad")
client.succeed("ob -v thunk pack ~/code/myapp --public")
client.fail("nix-build ~/code/myapp")
client.succeed("ob -v thunk unpack ~/code/myapp")
client.succeed("cd ~/code/myapp && git checkout master")
with subtest("test obelisk can detect private repos"):
client.succeed("ob -v thunk pack ~/code/myapp")
client.succeed("""grep -qF '"private": true' ~/code/myapp/git.json""")
client.succeed("ob -v thunk unpack ~/code/myapp")
'';
}) {}
Loading

0 comments on commit 57fd855

Please sign in to comment.