Anonymous functions or Smalltalk-style arguments?

Nice idea, I like it so far.

But is this something specific for passing references to functions or does it extend to all parameters allowing you to do soething like:

calculate() x 10.0 y 20.0

Quintesse wrote on May 23, 2011 14:26:
Nice idea, I like it so far. But is this something specific for passing references to functions or does it extend to all parameters

Good question. In theory we could go either way. The decision pivots on what this means:

when (x>100.0) then (100.0) otherwise (x)

Does it mean:

when (x>100.0, 100.0, x)

or does it mean:

when (x>100.0, () (100.0), () (x))

If we go with the first interpretation, then it is the full method invocation protocol of Smalltalk: you can pass values or anonymous functions. But we would be back to writing stuff like:

assert ("x must be positive") that () (x>0.0)

Which has more punctuation that I like.

If we go with the second interpretation, then we lose the ability to pass ordinary values as a Smalltalk-style argument, but we get a cleaner syntax for the usecases we're really interested in:

assert ("x must be positive") that (x>0.0)

Now, in theory, it would be possible to have the compiler decide between the two interpretations in the type analyzer rather than in the parser, so that (x>0.0) could be passing a value or a closure depending upon the declared type of the parameter that. But this kind of context-sensitive behavior is something I'm not a fan of in other languages.

Good question. In theory we could go either way. The decision pivots on what this means:

when (x>100.0) then (100.0) otherwise (x)

Does it mean:

when (x>100.0, 100.0, x)

or does it mean:

when (x>100.0, () (100.0), () (x))

Can't it mean:

when (x>100.0, (100.0), (x))

Which in trun is short hand for:

when (x>100.0, () (100.0), () (x))

Which would mean that

calculate() x 10.0 y 20.0 

is the same as:

calculate{ 
    x=10.0;
    y=20.0;}

and

calculate(10.0, 20.0)

But I must say that the Smalltalk style really provides excellent readability and is one of the features that was easy to get right away, simply because it reads so close to natural language. However when using it with values instead of functions it does not read as good at all.

Also I feel that allowing the traditional lambda style helps to explain and understand the smalltalk style, even if it does not provide better readability.

So my vote goes for supporting both, but only for functions not for values. Unless someone can provide examples where that also provides good readability..

@Sjur: the smalltalk syntax would make for very readable, almost DSL-like readable code:

divide(10) by 5

or something like

draw(LINE) from Point(1,5) to Point(7,23)

Ok, I haven't come up with the best of examples, but imagine does come up with some good ideas, they might just decide to do it anyway:

draw(LINE) from (Point(1,5)) to (Point(7,23))

and make from and to accept function references as well as values directly. This would make the implementing code uglier and there's probably be some overhead.

And making it possible to use the curly braces for the single-expression case as well would be out of the question, wouldn't it?

Well, the thing is that it would change the whole meaning of curly braces in the language. In Ceylon, like Java or C, a brace-delimited block doesn't have a value unless you explicitly return something. Indeed, an expression like x>0.0 is not even a valid statement or named argument. Yes, there are some languages where every expression is a statement, and the value of a block is the value of the last expression in the block, but I'm not a huge fan of this. I like the explicit return statement.

It's easy to miss the extra parentheses if you're just scanning the code.

I agree. Now, we could fix this by adding a keyword:

assert("x must be positive",function()(x>0.0))

You could even reuse local and void, without loss of regularity:

assert("x must be positive",local()(x>0.0))

But this is just getting even more verbose and making me like the anonymous function approach even less.

of course IDEs might help with color coding

Well, I don't think so - what precisely would they color code?

while curly braces make it a bit more obvious

Rather than breaking the regularity of the language, or ending up with a funny-looking syntax for anonymous functions, I would rather simply not have them, and only support them in Smalltalk-style arguments.

the smalltalk syntax would make for very readable, almost DSL-like readable code

Yes, that's a potential advantage. What I have not decided is if it would be a good thing or not. I'm finding a lot of the DSLs people are creating in other languages these days to be just, well ... kinda ugly. Worse, the intended grammar of these languages can be quite hard to tease out. It tends to be presented as a list of examples, not as a proper BNF.

The pursuit of code which reads like English is to me a fools errand. Natural languages are simply not appropriate for mechanized parsing, and the grammar of a natural language is not amenable to formal specification.

Instead, in Ceylon, we have our declarative syntax, which is intended as a much more disciplined way to solve some of the same kinds of problems that DSLs can be used for.

@Quintesse: Thanx for that. Looks really good. I change my vote!

Yes, that's a potential advantage. What I have not decided is if it would be a good thing or not. I'm finding a lot of the DSLs people are creating in other languages these days to be just, well ... kinda ugly. Worse, the intended grammar of these languages can be quite hard to tease out. It tends to be presented as a list of examples, not as a proper BNF.

I can agree with the general idea of DSL being to much, but in this specific case you will be giving developers a tool to transform potentially hard to read code into really good looking, easy to read code.

But like Quintesse mentiones, it's not good if it also makes it easy to do mistakes.

But in the specific example he gives I'm not really getting how that could be a problem. If assert accepts both a function and a boolean:

assert("",x>1)

and

assert("",()(x>1))

Would return the same result.

If assert only accepts either a function or a boolean, one of the statements won't compile.

Btw, is

assert("", (x>1))

out of the question? And then also

repeat(n,{writeLine "Hello";})

I'm guessing it is, but just not quite seeing why.

I can agree with the general idea of DSL being to much, but in this specific case you will be giving developers a tool to transform potentially hard to read code into really good looking, easy to read code.

There's a lot of tradeoffs here, and I don't know what the precisely best thing is. I'm just raising the kinds of issues we will need to think through.

Btw, is

assert("", (x>1))

out of the question?

Yes, totally out of the question. (x>1) is a parenthesized expression. The parens here mean grouping. I don't want them to potentially mean something totally different, depending upon context.

And then also

repeat(n,{writeLine "Hello";})

I'm guessing it is, but just not quite seeing why.

The expression {writeLine "Hello";} constructs a sequence of type Void[] with one element.

I vote for anonymous functions only as Smalltalk-style arguments

Well, the thing is that it would change the whole meaning of curly braces in the language. In Ceylon, like Java or C, a brace-delimited block doesn't have a value unless you explicitly return something. Indeed, an expression like x>0.0 is not even a valid statement or named argument. Yes, there are some languages where every expression is a statement, and the value of a block is the value of the last expression in the block, but I'm not a huge fan of this. I like the explicit return statement.

Actually in my opinion the statement block is less ambiguous than the expression parentheses in this context, because if you look at if(){ code } you already know that code is not going to be executed in some cases. You are used to blocks meaning deferred conditional evaluation. Whereas for expression parentheses, with syntax like map(l) with (foo()) it looks like lazy evaluation of an expression rather than declaring a lambda. I can understand why you like the explicit return statement, and I happen to like the explicit lambda or function keyword :)

Short of that I tend to agree that in my opinion map(l) with {foo} is less ambiguous to me that this is a special code construct that happens to be deferred evaluation-like.

Well, I don't think so - what precisely would they color code?

The lambda or function keyword :)

I'm trying to figure out a way to "share" context when defining several functions as arguments to a higher order function. A bit like in CSharp LINQ or Haskell "do" notation. To give you an idea:

from(people) (Person p) select (p.name) where (p.age>18) orderBy (p.age)

One way would be to use a class. So from the following definitions:

class Query<Y>(Y select, Boolean where = true) {...}
class OrderedQuery<Y, C>(Y select, C orderBy, Boolean where = true)
        extends Query<Y>(select, where)
        given C satisfies Comparable<C> {...}
Y[] from<Y, X>(X[] sequence, Query<Y> query(X x)) {...}

... you could do:

from(people)
OrderedQuery query(Person p) {
        local age = p.age;
        select = p.name;
        where = age>18;
        orderBy = age;
}

Would that parse? and compile?

I'm trying to figure out a way to "share" context when defining several functions as arguments to a higher order function.

Yeah, it's something I've also put thought into. I would like to be able to write:

from (people) select (Person p) (p.name) where (Person p) (p.age>18) by (Person p) (p.age)

As:

from (Person p in people) select (p.name) where (p.age>18) by (p.age)

It's a reasonably simply transformation in the type checker to make this work.

One way would be to use a class.

Very interesting idea. Never occurred to me. Here's how I would do it. I would leverage the declarative style of defining a method, i.e.:

Criteria<Person> on(Person p) {
    select = p.name;
    where = p.age>18;
    orderBy = p.age;
}

Now, in a Smalltak-style invocation, we're allowed to eliminate the return type, giving us:

from (people)
on (Person p) {
    select = p.name;
    where = p.age>18;
    orderBy = p.age;
};

I'm not 100% sure, but I think that would work w/o anything extra (unlike my solution). Good thinking!

Now, in a Smalltak-style invocation, we're allowed to eliminate the return type...

Are we also allowed not to eliminate it? What I am also looking for is a way to make the

orderBy

from (people)
on (Person p) {
    select = p.name;
    where = p.age>18;
};

... for the unordered search, and...

from (people)
OrderedCriteria<string, Natural> on (Person p) {
    select = p.name;
    where = p.age>18;
    orderBy = p.age;
};

... for the ordered search. This last one would be a bit less readable, though.

What I am also looking for is a way to make the orderBy optional, without recurring to a default value.

Surely the right way is this:

shared class Criteria<R, C>(R select, Boolean where=true, Comparable<C>? by = null) { ... }

shared class Criteria<R, C>(R select, Boolean where=true, Comparable<C>? by = null) { ... }

This would be enough for the user, but from the point of view of the implementor of the from method, you might want to be aware from the beginning that you can scrap the whole sorting business altogether... which you're not allowed to do if any call to your on parameter may give you a criteria with non-null by. I was thinking of something like

shared class Criteria<R, C>(R select, Boolean where = true, C by = null)
        given C of Nothing | Comparable<C> { ... }

but I don't think it makes sense. So the only solution might be to put no generic type constraint on C, and handle the situation at runtime:

• do not sort if C is Nothing,
• do sort if C is comparable
• throw an error otherwise.

No solution is perfect.

An eye-opening discussion on the utility of the object-instantiation syntax.

I am suddenly reminded of some minor questions about syntax.

1. How do you do a type union an object and a function? Is this the only way to do it (I guess it's not so bad...)?

void foo<T>(Bar<T>|Callable<Void,T> baz)

2. Do constructors allow the smalltalk-esque syntax? If so, it adds more regularity. Example. in the following, I don't know nor care if the build is a constructor or a plain-old function call:

Thing thing = build(maze)
with (Natural wood) {
    ...
};

(BTW, thanks for removing the new keyword)

3. What is the smalltalk-esque syntax when the function parameter is the only paramater? Are parens required?

all := ys.forAll() // with parens
every (Y) (y) {
    return y.isAwesome;
};

all := ys.forAll // without parens
every (Y) (y) {
    return y.isAwesome;
};

1. Yes, that looks correct.
2. Yes, they do. It's even quite useful.
3. Yes, the parens are required. I like them being there, since they make it very clear that a method is being called.

This would be enough for the user, but from the point of view of the implementor of the from method, you might want to be aware from the beginning that you can scrap the whole sorting business altogether

OK, now I get your point. Yes, the right solution would be to set C=Nothing. Since I have not figured out the details of the type argument inference algorithm yet, I can't say for sure if it will be smart enough to figure that out for itself. But I imagine it probably will be.

Here's my take on whether Smalltalk-like argument passing should allow for non-functional arguments: they shouldn't.

First reason is that you already have a quite nifty syntax for named arguments, so adding another - different - one might be confusing. The only exception is for function arguments. For these, Smalltalk-like syntax let you skip the return type declaration, and eventually the return statement, so they bring a real benefit.

Second reason relates to when every bit of code is evaluated. Both in traditional, position-based syntax, and declarative block syntax, the semantics are clear:

• Non-function arguments are evaluated before the call.
• The body of function arguments are evaluated any number of times during the call, but that is clearly spelled out by the way they are declared: as functions, complete with return type, argument list and return statement.

If you let Smalltalk-like arguments also apply to non-function values, then this behavior is undefined.

To anyone not aware of the mechanics of the language, Smalltalk-like function calls look like some kind of "new" control structures. Maybe this is the way they actually should think about them.

Gabriel, I'm inclined to agree with you.

Two humble suggestions (since we're at it). I have no idea how easy to implement those would be.

1. Declare the formal arguments (if any) to your argument-functions just before the in keyword, within your first pair of parentheses. You might have to add a requirement that all argument-functions have the same number and type of arguments. You would then have things like:

filter(String string in strings) by (!string.empty)

accumulate(Item item, Float sum in items, 0.0)
	using (sum+item.quantity*item.product.price)

from(Person p in people) 
	select (p.name) 
	where (p.age>18)
	orderBy (p.age)

As a bonus, your for construct is now implementable as a function.

2. Reuse this syntax as an alternative to the object keyword. I'll reuse my example:

shared interface Criteria<X, Y, C> {
	shared formal Y select(X x);
	shared formal Boolean where(X x);
	shared formal C orderBy(X x);
}

shared Y[] from<X, Y, C>(X[] sequence, Criteria<X, Y, C> criteria) { ... }

giving

from(Person p in people)
	select (p.name)
	where (p.age>18)
	orderBy (p.age)

How about...

shared interface ForControler<X> {
	shared formal void do(X x);
	shared default Boolean startingWhen(X x) { return true; }
	shared default Boolean while(X x) { return true; }
	shared default Boolean until(X x) { return false; }
	shared default void whenEmpty() { }
}

void for<X>(X[] sequence, ForControler<X> controler) { ... }

I gotta stop thinking about this.

That's a really interesting idea. I'll have to explore it further. The proposed feature I've written up in the language spec works like this:

You have two special annotations, iterated, and coordinated, that you use to declare the signature of the method:

shared List<Y> from<X,Y>(
    iterated Iterable<X> elements, 
    Boolean where(coordinated X x),
    Y select(coordinated X x)) { ... }

You can call the method like this:

List<String> names = from (Person p in people) where (p.age>18) select (p.name);

Which is equivalent to:

List<String> names = from (people) where (Person p) (p.age>18) select (Person p) (p.name);

But there may be some advantages to your idea.

Rethinking about it, you're probably right not to impose a single list of arguments to all argument-functions.

What would be great though would be to have a coherent set of "keywords" - most of them optional - to reuse in different constructs. Think of

for(Person p in people) orderBy (p.age) do {
	writeLn(p.age);
}

partition(Person p in people) where (p.age>18) by (p.age%10)

This way you can part with several one-trick pony combinators you'll find in many functional programming languages (filter on List giving out a List, sort on List giving out a List, etc.). One advantage is that with such a pattern, the user cannot accidentally put two where or two orderBy. And the implementor has more leeway to implement his sorting, filtering, grouping in the order and manner he wishes.

Gavin King wrote:

shared List<Y> from<X,Y>(
    iterated Iterable<X> elements, 
    Boolean where(coordinated X x),
    Y select(coordinated X x)) { ... }

and Gabriel Létourneau wrote on May 26, 2011 09:49:

for(Person p in people) orderBy (p.age) do {
        writeLn(p.age);
}

partition(Person p in people) where (p.age>18) by (p.age%10)

While I like this syntax, I worry that this is being limited to Iterables. Consider this:

ensure (Transaction t in connection) within {
   ...
}

Michael wrote on May 27, 2011 00:04:
While I like this syntax, I worry that this is being limited to Iterables. Consider this:

ensure (Transaction t in connection) within {
   ...
}

I agree in is rather indicative of collections. Perhaps : would be more neutral. I don't remember : being used as a keyword/operator in ceylon yet.

I was thinking of why you couldn't define if as a method in the like of

X if<X>(Boolean condition, X then(), X else())

so you could also use it as an expression, interpreting an absence of else as equivalent to else { } in the case where X = Void. But then I remembered your initialization checking feature, where

Boolean test;
if(x>0) {
	test = true;
}

... do stuff with test

would fail to compile because not all code paths ensure test is initialized. Same thing applies to for loops, where you can ensure initialization in the fail clause.

Imposing a more "functional" style might be an option here. After all you can re-write that example as

Boolean test = if (x>0) then (true) else (false);

I thought of a possible compromise, that would somehow limit your feature (if it's based on code-path analysis), but extend its use to all custom control structures.

Think of uninitialized declarations as formal parameters to the constructor of an implicit, anonymous, to-be-instantiated-right-away context-holding class. Then anything that comes right after can be thought of as an expression of that type, which can only be created through named-arguments syntax. So you could have things like

Boolean test;
if(x>0) then {
	test = true;
}
else {
	test = false;
}

Person? youngestLarry;
first(Person p in people) orderBy (p.age) where (p.name=="Larry") select {
	youngestLarry = p;
}
else {
	youngestLarry = null;
}

Of course, in the case of for loops you also lose the ability to use good old break and continue, but that is partly mitigated by :

• using return instead of continue, since we're really defining a method body.
• using different constructs (like my first).
• using additional controling clauses (like where).

Overly complex use of break and continue is bad practice anyway.

What would be great though would be to have a coherent set of "keywords" - most of them optional - to reuse in different constructs.

Yes, that's definitely what I'm thinking of :-)

While I like this syntax, I worry that this is being limited to Iterables.

Yes, you probably want a variation that looks like this:

String greeting = with (String name = process.arguments.first) then (name) otherwise ("World");

Person p = using (Session s = Session()) seek (s.get(Order, oid));

It works the same way, but the symbol is = instead of in. Again, these examples are taken out of a proposed feature section of the language spec. (That's where I put crazy ideas that I'm looking for feedback on.)

Gabriel, I spent a bunch of time trying to understand if there was a good way to define the built-in control structures in terms of higher-order functions, but in the end I concluded that the loss of occasionally useful capabilities (like non-local return, break, continue) that developers are used to from C, Java, C#, etc, along with the potential difficulties in conceptualizing and statically analyzing stuff like definite return and definite assignment, just wasn't worth the purity of going down this path. A major problem with this approach is that you would end up needing two kinds of return statement:

• a narrow-scoped statement, probably called yield, to return from the anonymous function, and
• a wider-scoped return statement to return from the containing declaration.

I really tried to convince myself that this path could work out, since I love how Smalltalk doesn't need control structures in the language definition. But in the end I decided that it just wouldn't be natural in this language, and is something that our target audience just isn't looking for.

But in the end I decided that it just wouldn't be natural in this language

A feature that is beautiful and elegant in one language, can look like a wart in a different language. You can't reduce a language to the intersection of its parts.

Gavin King wrote on May 29, 2011 00:33:
Gabriel, I spent a bunch of time trying to understand if there was a good way to define the built-in control structures in terms of higher-order functions, ...

I'm sure you did. Initialization checking is a killer feature in any situation. I can only hope you'll eventually find a way to extend it to custom control structures...

Gavin King wrote on May 29, 2011 00:20:

String greeting = with (String name = process.arguments.first) then (name) otherwise ("World");

Person p = using (Session s = Session()) seek (s.get(Order, oid));

I don't like these examples at all: What is the scope of name and s? All parameter lists of the function? Just those parameter lists following the one in which the value is introduced?

Can I write:

String greeting = use (name) otherwise ("World") with (String name = process.arguments.first);

?

Gabriel Létourneau wrote on May 29, 2011 18:21:
Initialization checking is a killer feature in any situation. I can only hope you'll eventually find a way to extend it to custom control structures...

Perhaps as an annotation-based compiler feature? Say, covered and branch :

shared covered Y find<X, Y>(
	Iterable<X> sequence,
	Boolean where(X x),
	branch Y select(X x),
	branch Y else()) {

	for(x in sequence) {
		if(where(x))
		{
			return select(x);
		}
	}
	fail {
		return else();
	}

}

The compiler would have to ensure that one and only one branch annotated parameter gets evaluated. Then you could rely on that on the call site to ensure initialization.

Perhaps as an annotation-based compiler feature?

I imagine you might be able to make that work.

What is the scope of name and s? All parameter lists of the function?

Parameters annotated coordinated.

Can I write:

String greeting = use (name) otherwise ("World") with (String name = process.arguments.first);

No, definitely not. The Type var = expression bit would always belong at the start of the positional argument list, it would not be part of the smalltalk argument list.

Gavin King wrote on May 30, 2011 13:07:
Perhaps as an annotation-based compiler feature? I imagine you might be able to make that work.

Perhaps, perhaps. I do have big dreams. Can't wait to get my hand on one branch of your compiler code!