Aquiles Issue Tracker Rss Feed

Closed Issue: divide by zero in SizeControlledConnectionPool.controlIdleClientSizeMethod [11340]

javiercanillas — Thu, 16 May 2013 10:32:03 GMT

I was testing the library with a basic configuration (see below) and my server reported a divide by zero exception in SizeControlledConnectionPool.controlIdleClientSizeMethod. I think it's because I did not specify the magicNumber in the config and the defaults aren't being properly applied. From the code it looks like it only sets the default values on a parse failure and not if the key/value does not exist in the SpecialConnectionParameterCollection. Once I added the magicNumber to the config it worked fine.

Also, it wasn't a problem but you might consider moving the creation of the TimerTask for controlIdleClientSizeMethod to the Initialize method to guarantee that all the settings have been initialized before the first time the TimerTask is triggered.

<aquilesConfiguration>

<clusters>
<add friendlyName="Service">
<connection poolType="SIZECONTROLLEDPOOL" factoryType="BUFFERED" />
<endpointManager type="ROUNDROBIN" defaultTimeout="6000">
<cassandraEndpoints>
<add address="RT3" port="9160"/>
</cassandraEndpoints>
</endpointManager>
</add>
</clusters>
</aquilesConfiguration>

Closed Issue: DeleteCommand validation exception when column value is null [11456]

javiercanillas — Thu, 16 May 2013 10:32:03 GMT

When using a DeleteCommand the input column must contain a not null value even though it is not actually needed for the deletation operation.

Closed Issue: BatchMutateCommand with SuperColumns over a Standard ColumnFamily [11744]

javiercanillas — Thu, 16 May 2013 10:32:03 GMT

BatchMutateCommand does not validate the ColumnFamily Type before proceeding with the execution. Doing so, you can try to push an insert with a SuperColumn over a Standard SuperColumn.

This is also enable when using BatchMutateCommand with DeletationMutation.

Suppose the following code:

Guid id = Guid.NewGuid();
BatchMutateCommand insertCMD = new BatchMutateCommand();
insertCMD.KeySpace = "Keyspace1";
Dictionary<string, Dictionary<string, List<IAquilesMutation>>> keyMutations = new Dictionary<string, Dictionary<string, List<IAquilesMutation>>>();
Dictionary<string, List<IAquilesMutation>> columnMutation = new Dictionary<string, List<IAquilesMutation>>();
List<IAquilesMutation> mutations = new List<IAquilesMutation>();

AquilesSetMutation insertContextDataMutation = new AquilesSetMutation();
insertContextDataMutation.SuperColumn = new AquilesSuperColumn()
{
Name = "Data",
Columns = new List<AquilesColumn>()
};

insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), DateTime.UtcNow));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), DateTime.UtcNow));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), 0));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), false));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), 0));

mutations.Add(insertContextDataMutation);

columnMutation.Add("Standard1", mutations);
string key = id.ToString();
keyMutations.Add(key, columnMutation);
insertCMD.Mutations = keyMutations;

using (IAquilesConnection connection = AquilesHelper.RetrieveConnection("TestCluster1"))
{
connection.Execute(insertCMD);
}

Closed Issue: MultiGetSliceCommand without SuperColumn over a Super ColumnFamily Type [11745]

javiercanillas — Thu, 16 May 2013 10:32:02 GMT

If you tried to execute a MultiGetSliceCommand without setting the SuperColumn (because you want to get all the child SuperColumns) over a ColumnFamily that is Super, you will get an exception with the message: "SuperColumn cannot be null or empty when accesing a Super type ColumnFamily.".

Suppose the following code:

Guid id = Guid.NewGuid();
BatchMutateCommand insertCMD = new BatchMutateCommand();
insertCMD.KeySpace = "Keyspace1";
Dictionary<string, Dictionary<string, List<IAquilesMutation>>> keyMutations = new Dictionary<string, Dictionary<string, List<IAquilesMutation>>>();
Dictionary<string,List<IAquilesMutation>> columnMutation = new Dictionary<string,List<IAquilesMutation>>();
List<IAquilesMutation> mutations = new List<IAquilesMutation>();

AquilesSetMutation insertContextDataMutation = new AquilesSetMutation();
insertContextDataMutation.SuperColumn = new AquilesSuperColumn()
{
Name = "Data",
Columns = new List<AquilesColumn>()
};

insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), DateTime.UtcNow));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), DateTime.UtcNow));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), 0));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), false));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), 0));

mutations.Add(insertContextDataMutation);

columnMutation.Add("Super1", mutations);
string key = id.ToString();
keyMutations.Add(key, columnMutation);
insertCMD.Mutations = keyMutations;

MultiGetSliceCommand multiGetSliceCMD = new MultiGetSliceCommand() { KeySpace = "Keyspace1", ColumnFamily = "Super1" };

AquilesSliceRange range = new AquilesSliceRange();
range.FinishColumn = DateTime.Now.ToUniversalTime().Ticks.ToString();
range.Reversed = true;
range.Count = int.MaxValue;

AquilesSlicePredicate predicate = new AquilesSlicePredicate();
predicate.SliceRange = range;

multiGetSliceCMD.Keys = new List<string>() { key };
multiGetSliceCMD.Predicate = predicate;

using (IAquilesConnection connection = AquilesHelper.RetrieveConnection("TestCluster1"))
{
connection.Open();
connection.Execute(insertCMD);
connection.Execute(multiGetSliceCMD);
connection.Close();
}

Assert.IsNotNull(multiGetSliceCMD.Output);

Closed Issue: GuidGenerator generates same Guid if called fast [12557]

apack12 — Thu, 16 May 2013 10:32:02 GMT

I try to generate multiple timeuuid using GuidGenerator.cs, but looks like it can generate duplicate uuid values sometimes.

Here is my code:

for (int i = 0; i < 10; i++)
{
Guid guid = Aquiles.Helpers.GuidGenerator.GenerateTimeBasedGuid();
Console.WriteLine(guid.ToString());
}

Here is the result:

7fda8f95-f444-11df-af7e-2cbfe486a130
7fdaddb6-f444-11df-af7e-2cbfe486a130
7fdaddb6-f444-11df-af7e-2cbfe486a130
7fdaddb6-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130

Is it a problem? Some of my CFs are using TimeUUIDType, if there are some insert actions at the same time, the uuid will be the same. Any solutions? Thanks.

Posted by: xamlguide on discussions

Closed Issue: GetKeyRangeSliceCommand doesn't support for empty Start and End Key [12796]

javiercanillas — Thu, 16 May 2013 10:32:02 GMT

To get all the entries in a columnfamily, cassandra expects the first startkey value to have StartKey.Length == 0. Current code doesn't allow this need something like the following:

1. Change AquilesKeyRange.cs ValidateNullOrEmptyStartKey to something like

private void ValidateNullOrEmptyStartKey()
{
// StartKey length == 0 means get the first available item
if (this.StartKey == null)
{
throw new AquilesCommandParameterException("StartKey must not be null.");
}
}

2. Also need to make an empty EndKey allowable - so something like

private void ValidateNullOrEmptyEndKey()
{
// EndKey can be null - means get as many as allowed by Count
}

3. Also need a type converter for IAquilesKeyTokenRange implementations - in ModelConverterHelper.cs around line 78 something like:

//Aquiles IAquilesKeyTokenRange converter
converter = new AquilesTokenRangeConverter();
converters.Add(typeof(AquilesTokenRange), converter);
converters.Add(typeof(AquilesKeyRange), converter);
converters.Add(typeof(KeyRange), converter);

4. Plus an implementation of the converter - something like:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Aquiles.Model;
using Apache.Cassandra;

namespace Aquiles.Converter.Model.Impl
{
/// <summary>
/// Converts from Aquiles.Model.IAquilesKeyTokenRange (Aquiles.Model.AquilesTokenRange) to Apache.Cassandra.KeyRange
///
/// </summary>
class AquilesTokenRangeConverter : IThriftConverter<IAquilesKeyTokenRange, KeyRange>
{
#region IThriftConverter<IAquilesKeyTokenRange,KeyRange> Members

public KeyRange Transform(IAquilesKeyTokenRange objectA)
{
if (objectA is AquilesTokenRange)
{
AquilesTokenRange tokenRange = (AquilesTokenRange)objectA;
return tokenRange.ToThrift();
//KeyRange result = new KeyRange
//{
// Count = tokenRange.Count,
// Start_token = tokenRange.StartToken,
// End_token = tokenRange.EndToken
//};
}
else if (objectA is AquilesKeyRange)
{
AquilesKeyRange keyRange = (AquilesKeyRange)objectA;
return keyRange.ToThrift();
//KeyRange result = new KeyRange
//{
// Count = keyRange.Count,
// Start_key = keyRange.StartKey,
// End_key = keyRange.EndKey
//};
}
else
throw new ArgumentException(string.Format("Don't know how to convert {0}", objectA));
}

public IAquilesKeyTokenRange Transform(KeyRange objectB)
{
if (objectB.End_token != null || objectB.Start_token != null)
{
return new AquilesTokenRange
{
StartToken = objectB.Start_token,
EndToken = objectB.End_token,
Count = objectB.Count
};
}
else
{
return new AquilesKeyRange
{
StartKey = objectB.Start_key,
EndKey = objectB.End_key,
Count = objectB.Count
};
}
}

#endregion
}
}

found by: patrickthompson
Discussion: http://aquiles.codeplex.com/Thread/View.aspx?ThreadId=240791

Closed Issue: UnavailableException and TimedOutException is not meaning of connection broken [12869]

javiercanillas — Thu, 16 May 2013 10:32:01 GMT

By reading again the Thrift Documentation, it seems that the UnavailableException and TimedOutException doesn't mean that the node you are connected is down or not healthy, so invaliding the connection and banning the endpoint is not a very clever idea.

I'm copying the part of the documention associated with it:

*UnavailableException*
Not all the replicas required could be created and/or read.
*TimedOutException*
The node responsible for the write or read did not respond during the rpc interval specified in your configuration (default 10s). This can happen if the request is too large, the node is oversaturated with requests, or the node is down but the failure detector has not yet realized it (usually this takes < 30s).

Commented Issue: ByteEncoderHelper.LongEncoder seems to expect reversed byte ordering to get correct value [15692]

Synaxin — Fri, 12 Oct 2012 02:43:17 GMT

I'm using the binary release of Aquiles 1.0.0.1, with a 1.1.5 cassandra cluster. Thanks for a great little API that is easy and straight forward to use.

When I use the cassandra-cli to insert a long value, 263752, to a column with a key A, as well as with the java client hector, that exact value is stored in cassandra. I can verify this easily with the cassandra-cli (see my attachment, 1st set of cassandra-cli commands)

When I retrieve that column in Aquiles, I have to reverse the order of the byte array before decoding to an integer using ByteEncoderHelper.LongEncoder. I haven't verified this with anything other than LongEncoder. (see attachment vb code)

When I insert a long with aquiles,I can also use cassandra-cli to view that same row, and it shows the aquiles inserted values as reversed. (see attachment, 2nd set of cassandra-cli commands)

Is this expected behavior with Aquiles? I can't be the first one to have seen this but I cannot find any issues, wiki/documentation, or google results which have this same issue.

I've attached a text file that you can use to reproduce the error with a very small # of lines so hopefully you will be able to reproduce this easily or tell me what I am doing wrong.
Comments: ** Comment from web user: Synaxin **

Just read v0.7.0.6, and used

test = ByteEncoderHelper.BigEndianLongEncoder.FromByteArray(l(0).Column.Value)

So a function exists to handle this already. It's not very clear to use this function from the documentation, the only way I found it was the change history for 0.7.0.6 (and intellisense once I looked).

I don't see why anyone would ever want anything other than BigEndian since cassandra is always going to store everything that way. But problem solved regardless, please close this out.

Commented Issue: ByteEncoderHelper.LongEncoder seems to expect reversed byte ordering to get correct value [15692]

Synaxin — Fri, 12 Oct 2012 01:54:54 GMT

Sorry for the rash of posts.

To cleanup the terminology of "reversed", when inserting a Long, if I first convert the long to NetworkOrder the big endian java server and java clients can read the long value. All aquiles needs to do is convert HostTonetwork when writing integers+ and convert NetworkTohost when reading integers+.

This code can be appended to the vb code in the text file.

test=System.net.IpAddress.HostToNetworkorder(test)
c.Value = ByteEncoderHelper.LongEncoder.ToByteArray(System.Net.IPAddress.HostToNetworkOrder(test))
m_cluster.Execute(Function(cl) cl.insert(key, cp, c, ConsistencyLevel.ONE), "TEST_KS")
l = m_cluster.Execute(Function(cl) cl.get_slice(key, cp, sc, ConsistencyLevel.ONE), "TEST_KS")
test = BitConverter.ToInt64(l(0).Column.Value, 0)
Console.WriteLine("BitConverter: " & test.ToString())
test = System.Net.IPAddress.NetworkToHostOrder(test)
Console.WriteLine("Network to Host order: " & test.ToString())

My values are correct in Cassandra-cli ,my java client, and in Aquiles if I convert network to host order after retrieving the column.

Right now I have a wrapper to use around ByteEncoderHelper.LongEncoder.To/FromByteArray and IntegerEncoder.To/FromByteArray

Commented Issue: ByteEncoderHelper.LongEncoder seems to expect reversed byte ordering to get correct value [15692]

Synaxin — Fri, 12 Oct 2012 01:19:06 GMT

Just to state the obvious,

If I write a long using Aquiles LongEncoder, and Read using Aquiles, there is no difference. It's the interoperability with other clients that is an issue.

and environments:
Client (aquiles, and hector) Machine was windows XP, 32 bit.
Server was CentOS 6.3, x86_64, cassandra 1.1.5 binary release .tar.gz

Created Issue: ByteEncoderHelper.LongEncoder seems to expect reversed byte ordering to get correct value [15692]

Synaxin — Fri, 12 Oct 2012 01:04:43 GMT

Closed Issue: When a node is down, and then restarted it will swap between blacklist and available list [14650]

javiercanillas — Wed, 11 Apr 2012 18:55:44 GMT

If there is a situation where a node went down, there will be still plenty of invalid connection on the pool. If the server goes up again pretty quick, then it is posible that the disconnected clients will attempt to put it back into blacklist, making the endpoint swap back and foward till all disconnected connections within the pool are cleansed.
Comments:

closed, fixed on 0.7.0.9, 0.8.3 and 1.0.0.2

Commented Issue: When a node is down, and then restarted it will swap between blacklist and available list [14650]

javiercanillas — Mon, 16 Jan 2012 17:55:42 GMT

fixed on revision 87413 and it will impact on 0.7.X, 0.8.X and 1.0.X

Created Issue: When a node is down, and then restarted it will swap between blacklist and available list [14650]

javiercanillas — Mon, 16 Jan 2012 17:54:28 GMT

Closed Issue: Suggestion: Int16 and unsigned encoder helpers [14392]

javiercanillas — Mon, 16 Jan 2012 17:51:00 GMT

Storing some data I've found that I'm having to do some casts back and forth between some smaller and unsigned types.

I could really use some more built in encoder helpers for Little/BigEndianInt16Encoder and for each integer encoder support (somehow) an unsigned and a signed version. I guess following the current convention this would be Little/BigEndianUInt16Encoder, and Little/BigEndianInt32Encoder.
Comments:

There is nothing to "fix"

Closed Issue: Exception: An existing connection was forcibly closed by the remote host [14528]

javiercanillas — Mon, 16 Jan 2012 17:50:33 GMT

hello, i encounter a problem when inserting a large value to a column, the value byte[] length is 25130744 (converted from DataSet to byte[]), and gives me an exception: IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
Anyone know the reason? Small column values work fine. Are there some data size limit or request timeout limit in Aquiles or Cassandra server?

Cassandra version: 1.0.2, Aquiles version: 1.0

Here is the detailed info:
Server Error in '/test' Application.

An existing connection was forcibly closed by the remote host

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host

Source Error:

Line 482: public void recv_insert()
Line 483: {
Line 484: TMessage msg = iprot_.ReadMessageBegin();
Line 485: if (msg.Type == TMessageType.Exception) {
Line 486: TApplicationException x = TApplicationException.Read(iprot_);

Source File: D:\codeplex\Aquiles\trunk\Aquiles.Cassandra10\Apache\Cassandra\Cassandra.cs Line: 484

Stack Trace:

[SocketException (0x2746): An existing connection was forcibly closed by the remote host]
System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) +245

[IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.]
System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) +7768653
Thrift.Transport.TStreamTransport.Read(Byte[] buf, Int32 off, Int32 len) in d:\thrift-0.6.0\thrift-0.6.0\lib\csharp\src\Transport\TStreamTransport.cs:84
Thrift.Transport.TTransport.ReadAll(Byte[] buf, Int32 off, Int32 len) in d:\thrift-0.6.0\thrift-0.6.0\lib\csharp\src\Transport\TTransport.cs:54
Thrift.Transport.TFramedTransport.ReadFrame() in d:\thrift-0.6.0\thrift-0.6.0\lib\csharp\src\Transport\TFramedTransport.cs:90
Thrift.Transport.TFramedTransport.Read(Byte[] buf, Int32 off, Int32 len) in d:\thrift-0.6.0\thrift-0.6.0\lib\csharp\src\Transport\TFramedTransport.cs:83
Thrift.Transport.TTransport.ReadAll(Byte[] buf, Int32 off, Int32 len) in d:\thrift-0.6.0\thrift-0.6.0\lib\csharp\src\Transport\TTransport.cs:54
Thrift.Protocol.TBinaryProtocol.ReadI32() in d:\thrift-0.6.0\thrift-0.6.0\lib\csharp\src\Protocol\TBinaryProtocol.cs:338
Thrift.Protocol.TBinaryProtocol.ReadMessageBegin() in d:\thrift-0.6.0\thrift-0.6.0\lib\csharp\src\Protocol\TBinaryProtocol.cs:220
Apache.Cassandra.Client.recv_insert() in D:\codeplex\Aquiles\trunk\Aquiles.Cassandra10\Apache\Cassandra\Cassandra.cs:484
DataAccessLib.CassandraAccess.<>c__DisplayClass28.<InsertCommand>b__27(Client client) in C:\TestProject\Class1.cs:746

[ExecutionBlockException: Exception 'Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.' during executing command. See inner exception for further details.]
Aquiles.Core.Cluster.Impl.DefaultCluster.Execute(Delegate executionBlock, String keyspaceName, ConnectionConfig overrideConnectionConfig) in D:\codeplex\Aquiles\trunk\Aquiles.Core\Cluster\Impl\DefaultCluster.cs:191
Comments:

Fixed. There is nothing wrong in the code.

Commented Issue: Exception: An existing connection was forcibly closed by the remote host [14528]

xamlguide — Mon, 12 Dec 2011 04:00:37 GMT

It works, thanks:)

Commented Issue: Exception: An existing connection was forcibly closed by the remote host [14528]

javiercanillas — Sat, 10 Dec 2011 03:54:54 GMT

I think your problem is that 25m is larger than 15 or even 16 that is the max support stream within Cassandra default configuration.

take a look at the following values inside cassandra.yaml:
# Frame size for thrift (maximum field length).
# 0 disables TFramedTransport in favor of TSocket. This option
# is deprecated; we strongly recommend using Framed mode.
thrift_framed_transport_size_in_mb: 15

# The max length of a thrift message, including all fields and
# internal thrift overhead.
thrift_max_message_length_in_mb: 16

Play around and let me know.

Created Issue: Exception: An existing connection was forcibly closed by the remote host [14528]

xamlguide — Fri, 09 Dec 2011 08:26:19 GMT

Commented Issue: Suggestion: Int16 and unsigned encoder helpers [14392]

ddaeschler — Tue, 15 Nov 2011 21:55:03 GMT

Hi Javier,

I haven't, I've been just casting back and forth which is a bit ugly. I can write them and submit the code after I'm done with my current piece in my project though.