2010-12-04  Dave Beckett  <dave@dajobe.org>

	* Snapshotted rasqal_0_9_21 for 0.9.21 release (GIT
	ec8c54cccd2cf2fdc716ac90dfb66f5d06b0a138)

	* INSTALL.html, README.html: update notes encouraging use of
	release tarballs, more using rasqal sections

	* docs/tmpl/section-service.sgml:
	Add section-service descriptions

	* utils/Makefile.am:
	Update for moved fix script

	* docs/.gitignore:
	Ignore rasqal-fake.i

	* docs/tmpl/section-data.sgml, docs/tmpl/section-expression.sgml,
	docs/tmpl/section-general.sgml,
	docs/tmpl/section-graph_pattern.sgml,
	docs/tmpl/section-literal.sgml, docs/tmpl/section-prefix.sgml,
	docs/tmpl/section-query.sgml,
	docs/tmpl/section-query_results_formatter.sgml,
	docs/tmpl/section-triples_source.sgml,
	docs/tmpl/section-variable.sgml: Update tmpls

	* Makefile.am, configure.ac, docs/Makefile.am, fix-groff-xhtml,
	scripts/Makefile.am, scripts/fix-bison, scripts/fix-flex,
	scripts/fix-groff-xhtml, scripts/fix-gtkdoc-header.pl,
	src/Makefile.am, src/fix-bison, src/fix-flex: Add scripts dir,
	move fix* scripts there and fixup gtkdoc-header with fake
	mechanism like raptor

2010-12-03  Dave Beckett <dave@dajobe.org>

	* src/rasqal.h.in, src/rasqal_internal.h, src/rasqal_raptor.c,
	src/rasqal_triples_source.c: Rename rasqal_triple_source_feature
	to rasqal_triples_source_feature to match other names

	* src/rasqal.h.in: autodocs

	* src/rasqal.h.in:
	Add RASQAL_API for rasqal_expression_compare

	* src/rasqal_literal.c: rasqal_literal_sequence_equals debug print
	less

	* src/rasqal.h.in: Declare RASQAL_RAPTOR_VERSION to record what
	version of raptor was built with.

	* configure.ac: Save raptor version that rasqal was built with in
	RAPTOR_VERSION_DEC

2010-12-02  Dave Beckett <dave@dajobe.org>

	* src/rasqal_datetime.c: internal docs

	* src/rasqal_datetime.c: code style

	* tests/sparql/aggregate/Makefile.am: Add group-concat-4 files to
	dist

	* tests/sparql/aggregate/Makefile.am,
	tests/sparql/aggregate/group-concat-4.rq,
	tests/sparql/aggregate/group-concat-4.ttl,
	tests/sparql/aggregate/manifest.n3: Add group-concat-4* test for
	GROUP DISTINCT

	* src/rasqal_rowsource_aggregation.c: Initialise group DISTINCT at
	start/end of each group

	(rasqal_aggregation_rowsource_init): Remove init of distinct map

	(rasqal_aggregation_rowsource_read_row): Destroy/create a distinct
	map at the group boundaries.

	* tests/sparql/aggregate/group-concat-2.rq: Test GROUP_CONCAT()
	with default separator

	* configure.ac, tests/algebra/Makefile.am,
	tests/engine/Makefile.am, tests/laqrs/syntax/Makefile.am,
	tests/rdql/Makefile.am, tests/rdql/testsuite/Makefile.am,
	tests/sparql/Expr1/Makefile.am, tests/sparql/Expr2/Makefile.am,
	tests/sparql/ExprBuiltins/Makefile.am,
	tests/sparql/ExprEquals/Makefile.am,
	tests/sparql/SyntaxDev/Syntax-SPARQL/Makefile.am,
	tests/sparql/SyntaxDev/Syntax-SPARQL2/Makefile.am,
	tests/sparql/SyntaxDev/Syntax-SPARQL3/Makefile.am,
	tests/sparql/SyntaxFull/Makefile.am,
	tests/sparql/ValueTesting/Makefile.am,
	tests/sparql/aggregate/Makefile.am,
	tests/sparql/bound/Makefile.am, tests/sparql/bugs/Makefile.am,
	tests/sparql/examples/Makefile.am,
	tests/sparql/federated/Makefile.am,
	tests/sparql/part1/Makefile.am, tests/sparql/regex/Makefile.am,
	tests/sparql/simple/Makefile.am, tests/sparql/sort/Makefile.am,
	tests/sparql/sparql11/Makefile.am,
	tests/sparql/survey/Makefile.am, tests/sparql/syntax/Makefile.am,
	tests/sparql/update/Makefile.am: Use local echo subst vars RECHO*
	and real echo program

	* src/rasqal_rowsource_aggregation.c: Add DISTINCTing of aggregate
	expression results in aggregate expressions.

	Remove literal_seq from expr_data

	(rasqal_aggregation_rowsource_read_row): Use
	rasqal_expression_sequence_evaluate() to make a new sequence of
	literals each time.  If a duplicate is found, do not evaluate a
	step.

	* src/rasqal_expr.c, src/rasqal_internal.h,
	src/rasqal_rowsource_groupby.c, src/rasqal_rowsource_having.c:
	Make rasqal_expression_sequence_evaluate() return a new seq of
	literal each time

	(rasqal_expression_sequence_evaluate): Lose literals_seq argument
	This was an unused and confusing approach.  A new list each time
	is easier to deal with.  Update all callers to
	rasqal_expression_sequence_evaluate()

	* src/rasqal_literal.c: Add tests for literal sequence sort map

2010-11-27  Dave Beckett <dave@dajobe.org>

	* src/rasqal_map.c:
	(rasqal_free_map_node): Free key / value only if there is a free
	handler.

	* src/rasqal_map.c:
	(rasqal_free_map_node): Free value properly

	* src/rasqal_internal.h: Fixes for building with raptor V1

	* src/rasqal_internal.h, src/rasqal_literal.c,
	src/rasqal_rowsource_aggregation.c: Added literal sequence sorting
	support

	(rasqal_new_literal_sequence_sort_map):
	Added

	(rasqal_literal_sequence_sort_map_add_literal_sequence): Added.

	* src/rasqal_literal.c:
	(rasqal_literal_sequence_equals): Added

	* src/Makefile.am, src/rasqal_internal.h, src/rasqal_literal.c:
	Add rasqal_literal_test

2010-11-25  Dave Beckett <dave@dajobe.org>

	* src/rasqal_algebra.c, src/rasqal_engine_sort.c,
	src/rasqal_internal.h, src/rasqal_map.c: Alter rasqal_map
	signature to take separate key/value free methods

	Add new typedef for raptor V1 for raptor_data_print_handler.

	(rasqal_new_map): Replaces rasqal_kv_free_fn free_fn arg with
	raptor_data_free_handler free_key_fn and free_value_fn.  Remove
	prototype ifdef for raptor V1/V2.

	Update all callers of rasqal_new_map().

	(rasqal_free_map_node): Pass in map arg down when freeing tree.

2010-11-19  Dave Beckett <dave@dajobe.org>

	* src/rasqal_engine_sort.c, src/rasqal_internal.h,
	src/rasqal_literal.c: Move literal array equals/compare functions
	to literal module

	(rasqal_literal_array_compare): Renamed from
	rasqal_engine_rowsort_compare_literals_sequence - takes two arrays
	not sequences.

	(rasqal_literal_array_equals): Renamed from
	rasqal_engine_rowsort_literal_sequence_equals() - takes two arrays
	not sequences.

	* src/Makefile.am, src/rasqal_algebra.c,
	src/rasqal_engine_algebra.c, src/rasqal_internal.h,
	src/rasqal_rowsource_having.c, tests/sparql/aggregate/agg-1.ttl,
	tests/sparql/aggregate/group-concat-3.rq,
	tests/sparql/aggregate/group-concat-3.ttl: Add support for SPARQL
	1.1 query HAVING

	rasqal_algebra_node_operator gains new algebra operator
	RASQAL_ALGEBRA_OPERATOR_HAVING

	rasqal_algebra_node updated field notes for having to use seq of
	expressions.

	(rasqal_new_having_algebra_node): Added.

	(rasqal_algebra_query_add_having): Added to add a having algebra
	node to the tree of algebra nodes

	(rasqal_algebra_having_algebra_node_to_rowsource): Added to turn a
	having algebra node into a having rowsource.

	(rasqal_algebra_node_to_rowsource): Call above for
	RASQAL_ALGEBRA_OPERATOR_HAVING.

	* src/rasqal_rowsource_groupby.c:
	(rasqal_groupby_rowsource_read_row): Bind result row to variables.

	* docs/tmpl/section-variable.sgml: update tmpls

2010-11-17  Dave Beckett <dave@dajobe.org>

	* src/rasqal_engine_algebra.c:
	(rasqal_query_engine_algebra_execute_init): Project before Order By

	* src/rasqal_rowsource_aggregation.c: Support COUNT(*) and
	COUNT(expr list)

	* src/rasqal.h.in: rasqal_variable gains usage field.

	* src/rasqal_algebra.c, src/rasqal_bindings.c, src/rasqal_query.c,
	src/rasqal_query_transform.c, src/rasqal_row.c,
	src/rasqal_rowsource.c, src/rasqal_rowsource_aggregation.c,
	src/rasqal_rowsource_project.c, src/rasqal_rowsource_triples.c,
	src/rasqal_variable.c, src/rdql_parser.y, src/sparql_parser.y:
	Switch rasqal_variable to sane reference counting model

2010-11-16  Dave Beckett <dave@dajobe.org>

	* src/rasqal_format_turtle.c, src/rasqal_internal.h,
	src/rasqal_literal.c: Improve turtle query results output

	(rasqal_literal_write_turtle): Added Turtle results format
	registration uses turtle mime type, no URI Only support variable
	bindings for Turtle Code style and tidy.

	* tests/sparql/check-sparql: More carefully make @vars_order from
	roqet debug output

	* tests/sparql/aggregate/Makefile.am: Add CLEANFILES

	* tests/sparql/aggregate/Makefile.am: Add SPARQL_TEST_FILES to
	dist

	* tests/sparql/aggregate/Makefile.am,
	tests/sparql/aggregate/agg-1.ttl,
	tests/sparql/aggregate/agg-2.ttl,
	tests/sparql/aggregate/agg-3.ttl,
	tests/sparql/aggregate/group-concat-1.ttl,
	tests/sparql/aggregate/group-concat-2.ttl,
	tests/sparql/aggregate/group-concat-3.ttl,
	tests/sparql/aggregate/manifest.n3: Add aggregate query execution
	tests

	* tests/sparql/check-sparql:
	Fix detecting variables order with
	bind-variables.  Also add some debug prints

	* src/Makefile.am, src/rasqal_format_turtle.c,
	src/rasqal_internal.h, src/rasqal_result_formats.c: Add turtle
	results format

	* tests/sparql/check-sparql: Add -i QUERY-LANGUAGE support

	* src/rasqal_rowsource_aggregation.c:
	(rasqal_aggregation_rowsource_read_row): Handle doing nothing if
	input rowsource is empty

	* src/rasqal_algebra.c, src/rasqal_rowsource_aggregation.c: Remove
	all uses of rasqal_new_variable_from_variable for now until
	variable reference counting is sane

	* src/rasqal_algebra.c:
	(rasqal_algebra_query_add_modifiers): Do nothing if there are no
	modifiers

	* src/rasqal_rowsource_aggregation.c:
	(rasqal_aggregation_rowsource_read_row): Bind agg value results to
	variables.

	* src/rasqal_algebra.c, src/rasqal_engine_algebra.c,
	src/rasqal_internal.h: Apply GROUP BY algebra node before
	aggregation, separate from other modifiers

	(rasqal_algebra_query_add_group_by): Added, pulled out of
	rasqal_algebra_query_add_modifiers

	(rasqal_query_engine_algebra_execute_init): Call above before
	aggregation node is added.

	* src/rasqal_algebra.c, src/rasqal_internal.h,
	src/rasqal_query_transform.c: Build variable use data and rebuild
	if query adds a variable in planning

	(rasqal_query_build_variables_use): Added.

	(rasqal_query_prepare_common): Call above

	(rasqal_algebra_query_prepare_aggregates): Call above when
	aggregate variables are added.

	* src/rasqal_query_transform.c:
	(rasqal_query_build_bound_in): Handle being called 2+ times

	* src/rasqal_query_transform.c:
	(rasqal_query_build_variables_use_map): Handle being called 2+
	times

2010-11-15  Dave Beckett <dave@dajobe.org>

	* src/rasqal_engine_algebra.c, src/rasqal_internal.h,
	src/rasqal_query.c, src/rasqal_rowsource_triples.c: Remove
	bound_in use from triples rowsource

	(rasqal_new_triples_rowsource): Lose bound_in and size args.

	(rasqal_algebra_basic_algebra_node_to_rowsource): Do not
	construct/copy bound_in from query variables.

	(rasqal_query_get_bound_in_column_for_variable): Added
	rasqal_rowsoruce_Triples_context loses bound_in and bound_in_size
	fields.

	(rasqal_triples_rowsource_init): Do not use above fields but call
	rasqal_query_get_bound_in_column_for_variable()

	* src/rasqal_algebra.c, src/rasqal_engine_algebra.c,
	src/rasqal_internal.h: Revert
	7038b5a52a555fa19d9cf07ff97a2981a3f5b68e and make tests pass again

2010-11-14  Dave Beckett <dave@dajobe.org>

	* src/rasqal_algebra.c, src/rasqal_engine_algebra.c,
	src/rasqal_internal.h: Move bound_in calculation outside algebra
	node

	(rasqal_algebra_basic_algebra_node_to_rowsource): Remove creation
	of bound_in to
	rasqal_algebra_make_bound_in_for_all_query_variables()

	(rasqal_algebra_make_bound_in_for_gp): Added containing calculaton
	for a graph pattern, taking into consideration variable use.

	(rasqal_algebra_make_bound_in_for_all_query_variables): Added
	providing old approach of assuming all selected variables are
	bound in triples.  Wrong.  rasqal_algebra_node gains bound_in and
	bound_in_size fields.

	(rasqal_new_triples_algebra_node): Add bound_in and bound_in_size
	args.

	(rasqal_algebra_basic_graph_pattern_to_algebra): Calculate
	bound_in for basic graph pattern before constructing triples
	algebra node.

	(rasqal_algebra_basic_graph_pattern_to_algebra): Free bound_in
	field.

	(main): Update test to create test bound_in.

	* src/rasqal_query_transform.c: Declare a variable bound in a
	triple only if unbound.

	(rasqal_query_triples_build_bound_in_internal): Assign column only
	if not already bound in a triple using test for BOUND_IN_UNBOUND.

2010-11-13  Dave Beckett <dave@dajobe.org>

	* src/rasqal_rowsource.c: more debug prints

	* src/rasqal_internal.h:
	Use unsigned 1 bit flags in structs rasqal_algebra_aggregate field
	adding_new_vars_is_error is unsigned
	1 bit flag rasqal_rowsource fields finished and generate_group are
	now unsigned 1 bit flags.

	* src/rasqal_algebra.c:
	(rasqal_algebra_extract_aggregate_expression_visit): Ignore
	non-agg exprs

	* src/rasqal_algebra.c:
	(rasqal_algebra_query_add_aggregation): Added

	* src/rasqal_algebra.c, src/rasqal_internal.h:
	(rasqal_algebra_extract_aggregate_expressions): Add agg_vars_seq
	to store variables during aggregation preparation

	* src/rasqal_algebra.c: debug print

	* src/rasqal_rowsource_aggregation.c:
	(rasqal_new_aggregation_rowsource): Copy args exprs_seq and
	vars_seq

	* src/rasqal_engine_algebra.c:
	(rasqal_query_engine_algebra_execute_init): Tidy if
	rasqal_algebra_query_add_aggregation fails.

	* src/rasqal_rowsource_project.c:
	(rasqal_new_project_rowsource): copy input projection sequence.

	* src/rasqal_variable.c:
	(rasqal_variable_copy_variable_sequence): Do not free variables in
	seq.

	* src/rasqal_rowsource_join.c, src/rasqal_rowsource_triples.c,
	src/rasqal_rowsource_union.c:
	autodocs

	* src/rasqal_algebra.c, src/rasqal_engine_sort.c,
	src/rasqal_expr.c, src/rasqal_internal.h, src/rasqal_query_write.c,
	src/rasqal_rowsource_groupby.c, src/rasqal_rowsource_rowsequence.c:
	
	Code style: use exprs_seq and vars_seq for sequences of expressions
	and variables

	* src/rasqal_internal.h, src/rasqal_variable.c:
	(rasqal_variable_copy_variable_sequence): Added

	* src/rasqal_internal.h:
	docs

	* src/rasqal_rowsource_assignment.c,
	src/rasqal_rowsource_distinct.c, src/rasqal_rowsource_empty.c,
	src/rasqal_rowsource_filter.c, src/rasqal_rowsource_graph.c,
	src/rasqal_rowsource_groupby.c, src/rasqal_rowsource_project.c,
	src/rasqal_rowsource_sort.c, src/rasqal_rowsource_triples.c:
	
	autodocs

2010-11-12  Dave Beckett <dave@dajobe.org>

	* src/rasqal_algebra.c:
	(rasqal_algebra_extract_aggregate_expression_visit): Add key:exp,
	value:var

	* src/rasqal_algebra.c:
	(rasqal_algebra_query_prepare_aggregates): Tidy on failures Free
	ae with rasqal_free_algebra_aggregate if things fail.

	* src/rasqal_expr.c:
	(rasqal_expression_convert_aggregate_to_variable): Malloc right
	size.

	* src/rasqal_rowsource_aggregation.c: Pass through/sample scalar
	variable values to output row rasqal_aggregation_rowsource_context
	gains a sequence of input_values, a counter and a step counter to
	know when to sample input rows for scalars.

	(rasqal_aggregation_rowsource_init,
	rasqal_aggregation_rowsource_finish): Init and free the new
	sequence. Reset the step count.

	(rasqal_aggregation_rowsource_ensure_variables): Copy in all input
	variables and record the count/offset in input_values_count for
	use later.

	(rasqal_aggregation_rowsource_read_row): If at first row of group,
	copy in all values into input_values sequence for use in
	generating the output row.  When generating output row, use those.

	(main): Update tests to expect input vars/values in output rows

2010-11-11  Dave Beckett <dave@dajobe.org>

	* src/rasqal_query_transform.c:
	(rasqal_query_build_variables_use_map): use row offsets correctly.

2010-11-08  Lauri Aalto <laalto@iki.fi>

	* src/rasqal_rowsource_aggregation.c:
	(rasqal_builtin_agg_expression_execute_step): fixed warning:
	result may be used before being set

2010-11-07  Dave Beckett <dave@dajobe.org>

	* src/rasqal_algebra.c, src/rasqal_engine_algebra.c,
	src/rasqal_internal.h: Return rasqal_algebra_aggregate to engine

	(rasqal_algebra_query_prepare_aggregates): return
	rasqal_algebra_aggregate.

	(rasqal_free_algebra_aggregate): Export internally.

	(rasqal_query_engine_algebra_execute_init): Update
	call to above and free returned structure.

	* src/rasqal_algebra.c: When looking for aggregate expressions,
	preserve scalar ones

	(rasqal_algebra_extract_aggregate_expression_visit): Add scalar
	exprs to agg_exprs list.

	(rasqal_algebra_query_prepare_aggregates): Debug message when no
	agg exprs found.

	* src/rasqal_algebra.c, src/rasqal_engine_algebra.c,
	src/rasqal_internal.h: Store sequence of expr + vars in algebra
	node for aggregation

	(rasqal_new_aggregation_algebra_node): Change params to be two
	sequences of expressions and variables.

	rasqal_algebra_node update docs for AGGREGATION using of sequence
	fields.

2010-11-06  Dave Beckett <dave@dajobe.org>

	* src/rasqal_rowsource_aggregation.c: docs

	* src/rasqal_rowsource_aggregation.c:
	(rasqal_aggregation_rowsource_finish): code style

	* src/rasqal_internal.h, src/rasqal_rowsource_aggregation.c:
	
	Aggregate rowsource now operates on seq of input expr/output vars

	rasqal_aggregation_rowsource_context gains sequence of expression
	input and sequence of output variables.

	(rasqal_aggregation_rowsource_finish): Free new seqs.

	(rasqal_new_aggregation_rowsource): Args now are the two sequences
	of expressions and variables.

	* src/rasqal_rowsource_aggregation.c: code style

	* src/rasqal_engine_algebra.c, src/rasqal_internal.h,
	src/rasqal_rowsource_aggregation.c: Remove func arg/field from
	aggregation rowsource and rasqal_algebra_node

	* src/rasqal_expr.c, src/rasqal_internal.h:
	(rasqal_expression_mentions_aggregate): Added

2010-11-05  Dave Beckett <dave@dajobe.org>

	* src/rasqal_rowsource_aggregation.c: Rename builtin_agg
	struct/methods to rasqal_builtin_agg_expression_execute*

	* src/rasqal_rowsource_aggregation.c:
	(rasqal_aggregation_rowsource_init): Request grouping.

	* src/rasqal_internal.h, src/rasqal_rowsource.c:
	Allow rowsource users to request groups

	rasqal_rowsource gains bit flag for marking request.  Turn
	finished flag into bitflag too while here.

	(rasqal_rowsource_request_grouping): Added to set flag.

	(rasqal_rowsource_read_row): Synthesise group if not in output row

	(rasqal_rowsource_read_rows): If seq is returned, check for group
	ID in first row and if missing, add to rest of rows.  If no seq is
	returned, filter each returned row and set a group ID if missing.

	* src/rasqal_rowsource_aggregation.c: Add rasqal_agg_expr_data for
	per-agg expr execution data.

	rasqal_agg_expr_data added with all data that is per-aggregate
	expression incuding input: expr, expr_seq (args) and output:
	variable to write to, sequence of literals and the per-execution
	data agg_user_data.

	rasqal_aggregation_rowsource_context loses a bunch of fields to
	above.  Refactor remaining code to add loops around per-expr data,
	only supporting 1-arg initialising at this time.

	* src/rasqal_rowsource_aggregation.c: Added
	rasqal_builtin_aggregation_reset to reset state to avoid
	finish/init pair

2010-11-04  Dave Beckett <dave@dajobe.org>

	* src/rasqal_expr.c:
	(rasqal_expression_clear): Do not try to clear name field for
	group_concat.

	* src/rasqal_rowsource_aggregation.c:
	(rasqal_aggregation_rowsource_read_row): Free builtin_agg data
	after result.

	* docs/tmpl/section-data.sgml, docs/tmpl/section-expression.sgml,
	docs/tmpl/section-graph_pattern.sgml,
	docs/tmpl/section-triples_source.sgml, src/rasqal_internal.h,
	src/rasqal_rowsource_groupby.c: Fixes for building with raptor V1

	* src/rasqal_algebra.c, src/rasqal_engine_algebra.c,
	src/rasqal_internal.h, src/rasqal_rowsource_aggregation.c: Alter
	rowsource aggregation calling convention to take an expression

	(rasqal_new_aggregation_rowsource): Take an expression; ditch op,
	parameters, flags.  Parameters will return later but is not used
	now for GROUP_CONCAT()

	rasqal_algebra_node loses unused/not used used fields op,
	parameters, flags.

	(rasqal_new_aggregation_algebra_node): Take an expression; ditch
	op, paramaters and flags.  Parameters will return later but is not
	used now for GROUP_CONCAT().  Creates or initialises an expression
	sequence based on the expression passed in.

	rasqal_aggregation_rowsource_context loses expr_seq_size, op, func
	and parameters fields.  Gains expr field.  builtin_agg gains expr
	field.

	(rasqal_builtin_aggregation_init): expr param replaces expr_seq,
	op, parameters.  Parameters will return later.

	(rasqal_aggregation_rowsource_finish): Destroy expr field.
	Updated test code to make/use an expression for agg testing.

	(rasqal_algebra_aggregation_algebra_node_to_rowsource): Update
	calling convention for aggregation rowsource.

2010-11-02  Dave Beckett <dave@dajobe.org>

	* src/rasqal_rowsource_aggregation.c:
	Made GROUP_CONCAT work with fixed ' ' separator.

	builtin_agg gains stringbuffer and separator

	(rasqal_builtin_aggregation_init): Init above for group concat.

	(rasqal_builtin_aggregation_finish): Free stringbuffer.

	(rasqal_builtin_aggregation_step): At each step, concat the
	literal string and put a separator before it if not first literal.

	(rasqal_builtin_aggregation_result): Compute result from
	stringbuffer.

	(main): Add group concat test

	* src/rasqal_engine_algebra.c:
	(rasqal_query_engine_algebra_execute_init): code tidy

	* src/rasqal_algebra.c, src/rasqal_engine_algebra.c,
	src/rasqal_internal.h: Pull out more algebra transformations and
	all take/return nodes

	(rasqal_algebra_query_add_modifiers): Take and return a node so
	that the algebra node tree gets modified.

	(rasqal_algebra_query_add_projection): Added to add projection
	node to algebra node tree.

	(rasqal_algebra_query_add_distinct): Added to add distinct node to
	algebra node tree.

	(rasqal_query_engine_algebra_execute_init): Call new functions
	above to transform algebra node tree.

	* tests/algebra/convert_graph_pattern.c:
	(main): Add new pulled-out query transforms.  code style

	* src/rasqal_internal.h: rasqal_algebra_aggregate gains error
	fields

	* src/rasqal_algebra.c:
	(rasqal_algebra_replace_aggregate_expressions): Added to transform
	expression sequences with aggregages.

	* src/rasqal_algebra.c:
	(rasqal_algebra_query_prepare_aggregates): Print agg expressions
	in debugging

	* src/rasqal_internal.h: rasqal_algebra_aggregate renames
	project_exprs field to agg_exprs and values are rasqal_expression

	* src/rasqal_algebra.c: Store aggregate expression in
	rasqal_algebra_aggregate

	agg_exprs seq rasqal_algebra_aggregate renames project_exprs field
	to agg_exprs and values are rasqal_expression

	(rasqal_algebra_extract_aggregate_expressions): Alter
	raptor_new_sequence args.

	* src/rasqal_internal.h: docs

	* src/rasqal_algebra.c, src/rasqal_internal.h: Introduce internal
	rasqal_algebra_aggregate

	(rasqal_free_algebra_aggregate): Added.

	(rasqal_algebra_query_prepare_aggregates): Dynamically create the
	rasqal_algebra_aggregate* and free with above.

	* src/rasqal_algebra.c, src/rasqal_engine_algebra.c,
	src/rasqal_internal.h:

	(rasqal_algebra_query_prepare_aggregates): Added pulled out of
	rasqal_algebra_query_to_algebra.

	* src/rasqal_algebra.c, src/rasqal_engine_algebra.c,
	src/rasqal_internal.h:

	(rasqal_algebra_query_add_modifiers): Pulled out of
	rasqal_algebra_query_to_algebra.

2010-11-02  Lauri Aalto <laalto@iki.fi>

	* src/rasqal_algebra.c:
	(rasqal_algebra_query_to_algebra): don't attempt to debugprint
	query->selects for construct queries

2010-11-01  Dave Beckett <dave@dajobe.org>

	* src/rasqal_algebra.c: Add initial code for transforming project
	expression to handle aggregates

	struct agg_extract added for handling state during walk of select
	expression trees.

	(rasqal_agg_expr_var_compare, rasqal_free_agg_expr_var): Added for
	support a rasqal_map of k:expression, v:variable when scanning for
	aggregate expressions.

	(rasqal_algebra_extract_aggregate_expression_visit): Added as a
	rasqal_expression tree visit function to do an in-line change of
	aggregate expressions to new internal variables.

	(rasqal_algebra_extract_aggregate_expressions): Added to wrap
	above transformation.

	(rasqal_algebra_query_to_algebra): Call above with initialised
	working state and print out result.  Free working state.

	* src/rasqal_expr.c, src/rasqal_internal.h:
	(rasqal_expression_convert_aggregate_to_variable): Added to do an
	inplace edit of an expression to replace an aggregate.  This feels
	like a hack but at least it's in one place

	* src/rasqal_expr.c, src/rasqal_internal.h:
	(rasqal_expression_is_aggregate): Added

	* src/rasqal.h.in, src/rasqal_expr.c:
	(rasqal_expression_compare): Added

	* src/rasqal_expr.c, src/rasqal_literal.c: Move
	rasqal_literal_sequence_compare to rasqal_literal.c

2010-10-31  Dave Beckett <dave@dajobe.org>

	* tests/sparql/aggregate/data-2.ttl: make ex:year data integer for
	later numeric testing

	* src/rasqal_algebra.c: code style

	* tests/sparql/aggregate/Makefile.am,
	tests/sparql/aggregate/agg-3.rq,
	tests/sparql/aggregate/data-2.ttl:
	
	Add agg-3.rq GROUP BY test with SAMPLE (lexes, parses, does not
	execute correctly)

	* src/sparql_lexer.l: Add SAMPLE

	* src/rasqal_algebra.c, src/rasqal_internal.h, src/rasqal_map.c,
	src/rasqal_rowsource_aggregation.c: Switch aggregation parameters
	from sequence to map

	Switch raptor_sequence* parameters to rasqal_map* parameters in
	aggregation rowsource and algebra node structures and functions.

	* src/rasqal_algebra.c, src/rasqal_engine_algebra.c,
	src/rasqal_internal.h: Start adding algebra nodes for grouping and
	aggregation

	rasqal_algebra_node_operator gains RASQAL_ALGEBRA_OPERATOR_GROUP
	and RASQAL_ALGEBRA_OPERATOR_AGGREGATE

	rasqal_algebra_node gains fields for aggregation: expr_op, func,
	parameters and flags.  group reuses seq.

	(rasqal_new_groupby_algebra_node,
	rasqal_new_aggregation_algebra_node): Added.
	rasqal_algebra_node_operator_labels gains labels for two new
	algebra operators.

	(rasqal_algebra_group_algebra_node_to_rowsource,
	rasqal_algebra_aggregation_algebra_node_to_rowsource): Added.

	(rasqal_algebra_query_to_algebra): Use
	rasqal_new_groupby_algebra_node() when group conditions expression
	is present to make a group algebra node. (Does not yet do
	aggregation).

	(rasqal_algebra_node_to_rowsource): Use
	rasqal_algebra_group_algebra_node_to_rowsource() and
	rasqal_algebra_aggregation_algebra_node_to_rowsource().

	* src/rasqal_rowsource_aggregation.c: docs

	* src/rasqal_map.c: code style

	* src/rasqal.h.in: Fix rasqal_literal_type arg in source header

	* src/rasqal_rowsource_aggregation.c:
	(rasqal_builtin_aggregation_step): Fix max/min compare

	(main): Add tests for final row outputs of aggregations

	* src/rasqal_literal.c: Fix rasqal_literal_write_type

	* src/rasqal.h.in, src/rasqal_literal.c:
	(rasqal_literal_type_label): Added and used for debug prints

2010-10-30  Dave Beckett <dave@dajobe.org>

	* src/rasqal_rowsource_aggregation.c:
	(rasqal_aggregation_rowsource_read_row): Create output row
	connected to rowsource.

	* src/rasqal_rowsource_aggregation.c: Add MIN, SUM, AVG, SAMPLE to
	tests

	* src/rasqal_row.c:
	(rasqal_new_row_sequence): Make integer literals from things that
	strtol() likes i.e. match ^[0-9]$

	* src/rasqal_rowsource_aggregation.c: Implement SAMPLE and AVG

	* src/rasqal_rowsource_aggregation.c: Aggregation works over
	multiple group inputs

	rasqal_aggregation_rowsource_context gains last_group_id to know
	boundaries, saved_row to store a row when a boundary change
	happens and offset to mark output rows correctly.

	(rasqal_aggregation_rowsource_init): Move agg init into read_row
	factory method.  Init last_group_id and offset.

	(rasqal_aggregation_rowsource_finish): Tidy up saved_row if left.

	(rasqal_aggregation_rowsource_read_row): Restructured to recognize
	start/end of group boundaries and end of input to know when to
	init/finish aggregation.  Otherwise, read input rows.

	(main): Make test expect 2 row results as it should be.

	* src/rasqal_rowsource_aggregation.c: Move literal_seq to a field
	of rasqal_aggregation_rowsource_context for reuse across groups.

	* src/rasqal_rowsource_aggregation.c:
	(rasqal_aggregation_rowsource_read_row): Evaluate expressions to
	form list of literals for aggregations step.

	* src/rasqal.h.in, src/rasqal_expr.c:
	(rasqal_expression_op_label): Added public function

	(rasqal_expression_write_op, rasqal_expression_print_op): Internal
	functions just call above.

	* src/rasqal_internal.h:
	(rasqal_new_aggregation_rowsource): Add flags arg.

	* src/rasqal_rowsource_aggregation.c:
	(rasqal_new_aggregation_rowsource): Add flags arg.

2010-10-28  Dave Beckett <dave@dajobe.org>

	* src/rasqal_internal.h, src/rasqal_rowsource_aggregation.c:
	Initial aggregation rowsource executes

	There are many issues to fix primarily in the execution steps to
	use the expression sequence and the issue of what to name the
	result variable, which the execution engine needs to decide.

	* src/rasqal_rowsource_aggregation.c,
	src/rasqal_rowsource_groupby.c: Implement get_inner_rowsource
	rowsource method for groupby and aggregation rowsources

	* src/Makefile.am, src/rasqal_internal.h,
	src/rasqal_rowsource_aggregation.c: Add skeleton aggregation
	rowsource with test that just copies rows through (cat rowsource)

	* src/rasqal_literal.c: less verbose when debugging

	* src/rasqal_rowsource_groupby.c: aargh

	* src/rasqal_format_table.c:
	(rasqal_query_results_write_table_bindings): Fix for raptor V2

	* src/rasqal_rowsource_groupby.c: Make group ID orders work for
	avltree ordering and raptor V1 sequence ordering

	* src/rasqal_rowsource_groupby.c:
	(raptor_avltree_search): compare correctly for raptor

	* src/rasqal_format_table.c:
	(rasqal_query_results_write_table_bindings): Remove unused world var

2010-10-28  Lauri Aalto <laalto@iki.fi>

	* src/rasqal_query_results.c:
	(rasqal_query_results_update_bindings): Do not access row values
	beyond row size

2010-10-27  Dave Beckett <dave@dajobe.org>

	* src/rasqal_rowsource_groupby.c:
	(main): Check number of groups returned is as expected

	* src/rasqal_rowsource_groupby.c: Add US Senators data test #3
	with 2 group by vars, many groups

	* src/rasqal_expr.c:
	(rasqal_literal_sequence_compare): Make less debug verbose

	* src/rasqal_row.c:
	(rasqal_new_row_sequence): Remove local reference to new literal.

	* src/rasqal_rowsource_groupby.c: Fix variable ownership and
	cleanup in tests

	* src/rasqal_rowsource_groupby.c: Fix groupby rowsource row
	returning to always return a new row

	(rasqal_groupby_rowsource_process): Do not copy row, let sequence
	own it.

	(rasqal_groupby_rowsource_read_row): Delete row from sequence so
	reference is passed back.

	* src/rasqal_rowsource_groupby.c:
	(Fake raptor_avltree_print): Return int on error

	* src/rasqal_rowsource_rowsequence.c:
	(rasqal_rowsequence_rowsource_finish): Free owned vars_seq

	* src/rasqal_rowsource_groupby.c: Memory
	tracking

	(rasqal_groupby_rowsource_finish): Ensure avltree iterator is
	destroyed

	(rasqal_groupby_rowsource_process): Copy/take reference to row
	before saving.

	* src/rasqal_rowsource_groupby.c: Refactor tests to be data driven

	* src/rasqal_rowsource_groupby.c: Print out grouping when computed
	and debugging

2010-10-26  Dave Beckett <dave@dajobe.org>

	* src/rasqal_rowsource_groupby.c: Add a fake rasqal_avltree to
	make groups work for Raptor V1

2010-10-25  Dave Beckett <dave@dajobe.org>

	* src/rasqal_internal.h: Added rasqal_row_bind_variables and fixed
	rasqal_new_groupby_rowsource()

	* src/rasqal_row.c:
	(rasqal_row_bind_variables): Added.

	* src/rasqal_rowsource_groupby.c: GROUP BY rowsource passes unit
	tests for no grouping / grouping

	(rasqal_groupby_rowsource_process): Using processed flag to know
	when to do no work so that when there are no expressions, the
	group ID is setup.  Use rasqal_row_bind_variables() to bind
	variables from the row into the variables table.  Add the node
	containing rows to the avltree in order to group them.

	(rasqal_groupby_rowsource_read_row): Handle no expression case as
	well as avltree/expression list case.

	(rasqal_new_groupby_rowsource): Add rowsource arg and save it.
	Tidy error cleanup path.

	(main): Add test case for no group by expression and one from
	SPARQL 1.1 query draft

	* src/rasqal_row.c: Set/show group IDs in rows

	(rasqal_new_row_common): Set default row group_id to
	-1 (rasqal_row_print): Print group_id if in range >= 0

	* src/rasqal_expr.c:
	(rasqal_expression_sequence_evaluate): Save a copy/free literal.

	* src/rasqal_expr.c:
	(rasqal_expression_sequence_evaluate): Do ignore errors when
	ignore_errors arg is set.

	(rasqal_literal_sequence_compare): Docs

	* src/rasqal_rowsource_groupby.c: Update groupby rowsource to use
	raptor_avltree for grouping

	Added typedef rasqal_groupby_tree_node to use as nodes in avltree.

	(rasqal_free_groupby_tree_node,
	rasqal_free_groupby_tree_print_node): Added.

	(rasqal_rowsource_groupby_literal_sequence_compare): Added to
	compare a sequence of literals in an avltree comparison.

