We're considering dropping pre-built Linux binaries


 

Hi all,

We're giving serious consideration to dropping the creation of pre-built Linux binaries are part of our release process. Some background:

We have two volunteers who do the majority of the release support. The amount of time that we invest in trying to debug and support installation across various Linux distributions is high. The amount of time we would need to do the job well is beyond our means as a volunteer-driven community project.

We currently build RPM and Deb packages for each release. We build these with LLVM 3.9 and a variety of other dependencies. As we've learned over the last few months, the difference between distros and their releases is quite high. Simply put: it's not something we are capable of taking on. We've tried and it hasn't been a success.

Our primary goal in providing binaries was to make it easier for people to try our Pony. Since we first starting doing regular Linux binary releases, we've added support for a Docker-based image that works on MacOS, Windows, and Linux. It's a great and simple way to get started with Pony as long as you have Docker installed. The amount of effort we need in order to maintain the Docker release in minimal. Additionally, we know have the online Pony Playground, http://playground.ponylang.org, that makes it easy to start trying Pony.

In general, we feel that Docker and Pony Playground are good tools for people "just trying out" Pony. From the State of the Stable survey that we will be publishing results from soon, about 1/3 of "long time" Pony users on Linux install via a package manager. That means that the majority of users are building from source.

Dropping attempts to try and support a variety of Linux platforms would give us more time to focus on building Pony features. It would also allow us to spend more time on documentation and other materials to help drive the Pony community forward. Rather than spending our weekends trying to figure out how to support various Linux distributions, we'd rather be writing new users' guides, Pony Patterns, fixing bugs and what not. We feel we can have a much bigger and more meaningful impact in that way.

The end goal has never been for us to maintain various Linux packages; for this to be sustainable, we need to have various distributions start packaging Pony. To that end, here are the two primary options we are considering:

- Limited Linux prebuilt binary support

In this scenario, we would support the latest long term Ubuntu release. 

- No Linux prebuilt binary support

In this scenario, we wouldn't provide any prebuilt Linux binaries.

In both scenarios, we'd be looking to members of the community to take over binary support. In the ideal scenario, packages for each distro would be part of that distro. We'd love to see Debian Pony users step up and becoming Debian package managers as part of the Debian project. The same holds true for other distributions. We'd work with package maintainers to make their jobs as easy as possible.

I'd personally like to thank everyone how has been involved in supporting the release process so far. 

- Kevin Cantu has stepped up and done a ton of work to get us this far. 
- Ștefan Talpalaru, thank you, for maintaining the Gentoo release. 

If you'd be willing to step forward and help, please let us know. 

Thanks,
Sean


 

On Sat, 2017-04-29 at 21:00 -0400, Sean T. Allen wrote:
Hi all,

We're giving serious consideration to dropping the creation of pre-
built
Linux binaries are part of our release process. Some background:
I note that https://www.ponylang.org/ doesn't have a "how to install
Pony" obviously anywhere.

The instructions on https://github.com/ponylang/ponyc imply that Linux
is a single thing line Windows or MacOS/OSX/macOS.

Windows and MacOS are (or seen as) two never inconsistent systems.
Linux is not equivalent. Linux is just the kernel. The equivalent of
Windows and MacOS is Debian, Fedora, Ubuntu, Mint, Arch, Gentoo, etc.
i.e. Linux is not a single thing it is a component in a Linux-based
distribution. The RPM and Deb install sections acknowledge this, but
then say (paraphrased) "these packages are rubbish", just build it
yourself from source.

Of course people coming to the GitHub page are likely set up and ready
to be able to build from source – well when LLVM 4 support is available
anyway. ;-)


We have two volunteers who do the majority of the release support.
The
amount of time that we invest in trying to debug and support
installation
across various Linux distributions is high. The amount of time we
would
need to do the job well is beyond our means as a volunteer-driven
community
project.
Hence, as you say below, getting pony into the Linux distributions,
which would be the way to go.

Alternatively follow the Rust route of having a really easy Bash based
system for downloading the latest release version and building locally.
It turns out to be a really successful and useful system. It means
supporting the means of delivery of the target rather than the target
itself.

We currently build RPM and Deb packages for each release. We build
these
with LLVM 3.9 and a variety of other dependencies. As we've learned
over
the last few months, the difference between distros and their
releases is
quite high. Simply put: it's not something we are capable of taking
on.
We've tried and it hasn't been a success.
Sorry about bringing that one up. :-)

Our primary goal in providing binaries was to make it easier for
people to
try our Pony. Since we first starting doing regular Linux binary
releases,
we've added support for a Docker-based image that works on MacOS,
Windows,
and Linux. It's a great and simple way to get started with Pony as
long as
you have Docker installed. The amount of effort we need in order to
maintain the Docker release in minimal. Additionally, we know have
the
online Pony Playground, http://playground.ponylang.org, that makes it
easy
to start trying Pony.
Whilst I have no experience of all this Docker stuff, it does provide a
level of indirection to separate your builds from the vagaries of the
Linux distribution milieu. It also plays into the DevOps folk hands.

In general, we feel that Docker and Pony Playground are good tools
for
people "just trying out" Pony. From the State of the Stable survey
that we
will be publishing results from soon, about 1/3 of "long time" Pony
users
on Linux install via a package manager. That means that the majority
of
users are building from source.
But beware using this "statistic" which is about the early adopters,
when in reality the process needs to play (eventually) to the more
general programmer world, and the deployment world. This doesn't
invalidate the decision now, but as Pony begins to gain traction, it is
a decision to revisit.

Dropping attempts to try and support a variety of Linux platforms
would
give us more time to focus on building Pony features. It would also
allow
us to spend more time on documentation and other materials to help
drive
the Pony community forward. Rather than spending our weekends trying
to
figure out how to support various Linux distributions, we'd rather be
writing new users' guides, Pony Patterns, fixing bugs and what not.
We feel
we can have a much bigger and more meaningful impact in that way.
On the other hand by providing Windows and MacOS per-built, but not a
route for any Linux based distributions, you are sending a message to
the world: we care about our Windows and MacOS users, you Linux users,
you go fend for yourself, we are not caring about you. This is a
standard trap, many horses in the race of language adoption have fallen
at the first hurdle of choosing to focus on one audience. (Allusion
intended. :-)

The end goal has never been for us to maintain various Linux
packages; for
this to be sustainable, we need to have various distributions start
packaging Pony. To that end, here are the two primary options we are
considering:
Yes, but there has to be a central correct system for all. Go and Rust
have found ways of dealing with "our platform packaging is 38 versions
behind the current release". The Go solution is tar-based, and a wee
bit annoying, the Rust solution of automating download and install with
Bash is surprising easy to use.

- Limited Linux prebuilt binary support

In this scenario, we would support the latest long term Ubuntu
release.
This only works if Ubuntu is the major target, which it might be, cf.
Swift. It turns out the Ubuntu build often works on Debian Sid anyway.
However the Fedora folk are being ignored.

- No Linux prebuilt binary support

In this scenario, we wouldn't provide any prebuilt Linux binaries.

In both scenarios, we'd be looking to members of the community to
take over
binary support. In the ideal scenario, packages for each distro would
be
part of that distro. We'd love to see Debian Pony users step up and
becoming Debian package managers as part of the Debian project. The
same
holds true for other distributions. We'd work with package
maintainers to
make their jobs as easy as possible.
D release binaries as Pony does now, at least DMD does, and they sort
of work, but then they have far less dependency on crucial things, such
as LLVM. LDC has been accepted into Debian and Fedora and so comes
platform packaged with the LLVM that comes with. This has to be the
goal. Go and Rust do it, but are sometimes behind the version game. D
and Go, as well as Pony, could learn from the Rust approach.

I'd personally like to thank everyone how has been involved in
supporting
the release process so far.

- Kevin Cantu has stepped up and done a ton of work to get us this
far.
- Ștefan Talpalaru, thank you, for maintaining the Gentoo release.

If you'd be willing to step forward and help, please let us know.
I am not going to be able to step up any time soon, which kind of makes
my words above a little "consultant". Sorry.
--
Russel.
=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder


 

Language communities at other maturity levels, like those you've mentioned, do often support quite a wide range of platforms, and that can be wonderful:
http://www.oracle.com/technetwork/java/javase/certconfig-2095354.html

In the far future I'm quite interested what operating systems will be run commercially on the hardware where Pony might have the best mechanical sympathy.  For example, Willem Wyndham has some fascinating ideas about the hardware, here:
https://vimeo.com/172129187

But then, as now, it'll be worth considering the best use of the community's time.


Kevin



On Sun, Apr 30, 2017 at 11:06 AM, Russel <russel@...> wrote:
On Sat, 2017-04-29 at 21:00 -0400, Sean T. Allen wrote:
> Hi all,
>
> We're giving serious consideration to dropping the creation of pre-
> built
> Linux binaries are part of our release process. Some background:

I note that https://www.ponylang.org/ doesn't have a "how to install
Pony" obviously anywhere.

The instructions on https://github.com/ponylang/ponyc imply that Linux
is a single thing line Windows or MacOS/OSX/macOS.

Windows and MacOS are (or seen as) two never inconsistent systems.
Linux is not equivalent. Linux is just the kernel. The equivalent of
Windows and MacOS is Debian, Fedora, Ubuntu, Mint, Arch, Gentoo, etc.
i.e. Linux is not a single thing it is a component in a Linux-based
distribution. The RPM and Deb install sections acknowledge this, but
then say (paraphrased) "these packages are rubbish", just build it
yourself from source.

Of course people coming to the GitHub page are likely set up and ready
to be able to build from source – well when LLVM 4 support is available
anyway. ;-)


> We have two volunteers who do the majority of the release support.
> The
> amount of time that we invest in trying to debug and support
> installation
> across various Linux distributions is high. The amount of time we
> would
> need to do the job well is beyond our means as a volunteer-driven
> community
> project.

Hence, as you say below, getting pony into the Linux distributions,
which would be the way to go.

Alternatively follow the Rust route of having a really easy Bash based
system for downloading the latest release version and building locally.
It turns out to be a really successful and useful system. It means
supporting the means of delivery of the target rather than the target
itself.

> We currently build RPM and Deb packages for each release. We build
> these
> with LLVM 3.9 and a variety of other dependencies. As we've learned
> over
> the last few months, the difference between distros and their
> releases is
> quite high. Simply put: it's not something we are capable of taking
> on.
> We've tried and it hasn't been a success.

Sorry about bringing that one up. :-)

> Our primary goal in providing binaries was to make it easier for
> people to
> try our Pony. Since we first starting doing regular Linux binary
> releases,
> we've added support for a Docker-based image that works on MacOS,
> Windows,
> and Linux. It's a great and simple way to get started with Pony as
> long as
> you have Docker installed. The amount of effort we need in order to
> maintain the Docker release in minimal. Additionally, we know have
> the
> online Pony Playground, http://playground.ponylang.org, that makes it
> easy
> to start trying Pony.

Whilst I have no experience of all this Docker stuff, it does provide a
level of indirection to separate your builds from the vagaries of the
Linux distribution milieu. It also plays into the DevOps folk hands.

> In general, we feel that Docker and Pony Playground are good tools
> for
> people "just trying out" Pony. From the State of the Stable survey
> that we
> will be publishing results from soon, about 1/3 of "long time" Pony
> users
> on Linux install via a package manager. That means that the majority
> of
> users are building from source.

But beware using this "statistic" which is about the early adopters,
when in reality the process needs to play (eventually) to the more
general programmer world, and the deployment world. This doesn't
invalidate the decision now, but as Pony begins to gain traction, it is
a decision to revisit.

> Dropping attempts to try and support a variety of Linux platforms
> would
> give us more time to focus on building Pony features. It would also
> allow
> us to spend more time on documentation and other materials to help
> drive
> the Pony community forward. Rather than spending our weekends trying
> to
> figure out how to support various Linux distributions, we'd rather be
> writing new users' guides, Pony Patterns, fixing bugs and what not.
> We feel
> we can have a much bigger and more meaningful impact in that way.

On the other hand by providing Windows and MacOS per-built, but not a
route for any Linux based distributions, you are sending a message to
the world: we care about our Windows and MacOS users, you Linux users,
you go fend for yourself, we are not caring about you. This is a
standard trap, many horses in the race of language adoption have fallen
at the first hurdle of choosing to focus on one audience. (Allusion
intended. :-)

> The end goal has never been for us to maintain various Linux
> packages; for
> this to be sustainable, we need to have various distributions start
> packaging Pony. To that end, here are the two primary options we are
> considering:

Yes, but there has to be a central correct system for all. Go and Rust
have found ways of dealing with "our platform packaging is 38 versions
behind the current release". The Go solution is tar-based, and a wee
bit annoying, the Rust solution of automating download and install with
Bash is surprising easy to use.

> - Limited Linux prebuilt binary support
>
> In this scenario, we would support the latest long term Ubuntu
> release.

This only works if Ubuntu is the major target, which it might be, cf.
Swift. It turns out the Ubuntu build often works on Debian Sid anyway.
However the Fedora folk are being ignored.

> - No Linux prebuilt binary support
>
> In this scenario, we wouldn't provide any prebuilt Linux binaries.
>
> In both scenarios, we'd be looking to members of the community to
> take over
> binary support. In the ideal scenario, packages for each distro would
> be
> part of that distro. We'd love to see Debian Pony users step up and
> becoming Debian package managers as part of the Debian project. The
> same
> holds true for other distributions. We'd work with package
> maintainers to
> make their jobs as easy as possible.

D release binaries as Pony does now, at least DMD does, and they sort
of work, but then they have far less dependency on crucial things, such
as LLVM. LDC has been accepted into Debian and Fedora and so comes
platform packaged with the LLVM that comes with. This has to be the
goal. Go and Rust do it, but are sometimes behind the version game. D
and Go, as well as Pony, could learn from the Rust approach.

> I'd personally like to thank everyone how has been involved in
> supporting
> the release process so far.
>
> - Kevin Cantu has stepped up and done a ton of work to get us this
> far.
> - Ștefan Talpalaru, thank you, for maintaining the Gentoo release.
>
> If you'd be willing to step forward and help, please let us know.

I am not going to be able to step up any time soon, which kind of makes
my words above a little "consultant". Sorry.
--
Russel.
=============================================================================
Dr Russel Winder      t: +44 20 7585 2200   voip: sip:russel.winder@...
41 Buckmaster Road    m: +44 7770 465 077   xmpp: russel@...
London SW11 1EN, UK   w: www.russel.org.uk  skype: russel_winder





 

With regard to the aim of making it easier to build from source on
Linux platforms - one major stumbling block toward that goal is our
current dependency on (a specific version of) `pcre`.

So if someone wanted to step up and help make Pony more accessible,
but preferred a code project rather than working with
packaging/building/deploying systems, they could work on a pure Pony
version of our `regex` package, so we could drop our `pcre`
dependency.

At that point, our main dependency would be LLVM, which Rust also has
- so we may even be able to copy (and properly attribute) their
build/install strategy more or less verbatim.

I'd also like to echo the sentiment that this is an all-volunteer
project, so while we can set goals about what we'd like to do and we
can strategize about how best to do it, we're ultimately at the mercy
of what parts of the project our volunteers are interested in working
on - each contributor scratching their own itches and exploring as
their own creativity guides them. Pressuring someone into working on
something they'd rather not work on when they aren't getting paid for
it is one of the quickest paths to contributor burnout, and that does
nobody any good. Unlike corporate-backed languages, we don't have
"resources to allocate" when it comes to developer time - we have a
federation of people with varying goals who are motivated to work with
Pony for their own reasons, and keeping those people
happy/motivated/interested is our key to continued progress.

So that's my general opinion. If we don't have anyone who is
passionate about (and has adequate time to contribute) supporting
pre-built Linux binaries, we can't feasibly support them. This is not
a value judgment about Linux with regard to Apple or Windows operating
systems (in fact I'm a happy Linux user and believe it's the best
platform for software developers) - it's just an assessment of where
we're at.

I'll close by saying that I'm continually inspired by this community's
ability to align together to collectively solve problems, and I'm
quite happy to be a part of it.

On Sun, Apr 30, 2017 at 10:32 PM, Kevin Cantu <kevincantu@gmail.com> wrote:
Language communities at other maturity levels, like those you've mentioned,
do often support quite a wide range of platforms, and that can be wonderful:
http://www.oracle.com/technetwork/java/javase/certconfig-2095354.html

In the far future I'm quite interested what operating systems will be run
commercially on the hardware where Pony might have the best mechanical
sympathy. For example, Willem Wyndham has some fascinating ideas about the
hardware, here:
https://vimeo.com/172129187

But then, as now, it'll be worth considering the best use of the community's
time.


Kevin



On Sun, Apr 30, 2017 at 11:06 AM, Russel <russel@winder.org.uk> wrote:

On Sat, 2017-04-29 at 21:00 -0400, Sean T. Allen wrote:
Hi all,

We're giving serious consideration to dropping the creation of pre-
built
Linux binaries are part of our release process. Some background:
I note that https://www.ponylang.org/ doesn't have a "how to install
Pony" obviously anywhere.

The instructions on https://github.com/ponylang/ponyc imply that Linux
is a single thing line Windows or MacOS/OSX/macOS.

Windows and MacOS are (or seen as) two never inconsistent systems.
Linux is not equivalent. Linux is just the kernel. The equivalent of
Windows and MacOS is Debian, Fedora, Ubuntu, Mint, Arch, Gentoo, etc.
i.e. Linux is not a single thing it is a component in a Linux-based
distribution. The RPM and Deb install sections acknowledge this, but
then say (paraphrased) "these packages are rubbish", just build it
yourself from source.

Of course people coming to the GitHub page are likely set up and ready
to be able to build from source – well when LLVM 4 support is available
anyway. ;-)


We have two volunteers who do the majority of the release support.
The
amount of time that we invest in trying to debug and support
installation
across various Linux distributions is high. The amount of time we
would
need to do the job well is beyond our means as a volunteer-driven
community
project.
Hence, as you say below, getting pony into the Linux distributions,
which would be the way to go.

Alternatively follow the Rust route of having a really easy Bash based
system for downloading the latest release version and building locally.
It turns out to be a really successful and useful system. It means
supporting the means of delivery of the target rather than the target
itself.

We currently build RPM and Deb packages for each release. We build
these
with LLVM 3.9 and a variety of other dependencies. As we've learned
over
the last few months, the difference between distros and their
releases is
quite high. Simply put: it's not something we are capable of taking
on.
We've tried and it hasn't been a success.
Sorry about bringing that one up. :-)

Our primary goal in providing binaries was to make it easier for
people to
try our Pony. Since we first starting doing regular Linux binary
releases,
we've added support for a Docker-based image that works on MacOS,
Windows,
and Linux. It's a great and simple way to get started with Pony as
long as
you have Docker installed. The amount of effort we need in order to
maintain the Docker release in minimal. Additionally, we know have
the
online Pony Playground, http://playground.ponylang.org, that makes it
easy
to start trying Pony.
Whilst I have no experience of all this Docker stuff, it does provide a
level of indirection to separate your builds from the vagaries of the
Linux distribution milieu. It also plays into the DevOps folk hands.

In general, we feel that Docker and Pony Playground are good tools
for
people "just trying out" Pony. From the State of the Stable survey
that we
will be publishing results from soon, about 1/3 of "long time" Pony
users
on Linux install via a package manager. That means that the majority
of
users are building from source.
But beware using this "statistic" which is about the early adopters,
when in reality the process needs to play (eventually) to the more
general programmer world, and the deployment world. This doesn't
invalidate the decision now, but as Pony begins to gain traction, it is
a decision to revisit.

Dropping attempts to try and support a variety of Linux platforms
would
give us more time to focus on building Pony features. It would also
allow
us to spend more time on documentation and other materials to help
drive
the Pony community forward. Rather than spending our weekends trying
to
figure out how to support various Linux distributions, we'd rather be
writing new users' guides, Pony Patterns, fixing bugs and what not.
We feel
we can have a much bigger and more meaningful impact in that way.
On the other hand by providing Windows and MacOS per-built, but not a
route for any Linux based distributions, you are sending a message to
the world: we care about our Windows and MacOS users, you Linux users,
you go fend for yourself, we are not caring about you. This is a
standard trap, many horses in the race of language adoption have fallen
at the first hurdle of choosing to focus on one audience. (Allusion
intended. :-)

The end goal has never been for us to maintain various Linux
packages; for
this to be sustainable, we need to have various distributions start
packaging Pony. To that end, here are the two primary options we are
considering:
Yes, but there has to be a central correct system for all. Go and Rust
have found ways of dealing with "our platform packaging is 38 versions
behind the current release". The Go solution is tar-based, and a wee
bit annoying, the Rust solution of automating download and install with
Bash is surprising easy to use.

- Limited Linux prebuilt binary support

In this scenario, we would support the latest long term Ubuntu
release.
This only works if Ubuntu is the major target, which it might be, cf.
Swift. It turns out the Ubuntu build often works on Debian Sid anyway.
However the Fedora folk are being ignored.

- No Linux prebuilt binary support

In this scenario, we wouldn't provide any prebuilt Linux binaries.

In both scenarios, we'd be looking to members of the community to
take over
binary support. In the ideal scenario, packages for each distro would
be
part of that distro. We'd love to see Debian Pony users step up and
becoming Debian package managers as part of the Debian project. The
same
holds true for other distributions. We'd work with package
maintainers to
make their jobs as easy as possible.
D release binaries as Pony does now, at least DMD does, and they sort
of work, but then they have far less dependency on crucial things, such
as LLVM. LDC has been accepted into Debian and Fedora and so comes
platform packaged with the LLVM that comes with. This has to be the
goal. Go and Rust do it, but are sometimes behind the version game. D
and Go, as well as Pony, could learn from the Rust approach.

I'd personally like to thank everyone how has been involved in
supporting
the release process so far.

- Kevin Cantu has stepped up and done a ton of work to get us this
far.
- Ștefan Talpalaru, thank you, for maintaining the Gentoo release.

If you'd be willing to step forward and help, please let us know.
I am not going to be able to step up any time soon, which kind of makes
my words above a little "consultant". Sorry.
--
Russel.

=============================================================================
Dr Russel Winder t: +44 20 7585 2200 voip:
sip:russel.winder@ekiga.net
41 Buckmaster Road m: +44 7770 465 077 xmpp: russel@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder