Aquiles

Reopened Issue: divide by zero in SizeControlledConnectionPool.controlIdleClientSizeMethod [11340]

javiercanillas — Fri, 14 Jun 2013 07:16:10 GMT

I was testing the library with a basic configuration (see below) and my server reported a divide by zero exception in SizeControlledConnectionPool.controlIdleClientSizeMethod. I think it's because I did not specify the magicNumber in the config and the defaults aren't being properly applied. From the code it looks like it only sets the default values on a parse failure and not if the key/value does not exist in the SpecialConnectionParameterCollection. Once I added the magicNumber to the config it worked fine.

Also, it wasn't a problem but you might consider moving the creation of the TimerTask for controlIdleClientSizeMethod to the Initialize method to guarantee that all the settings have been initialized before the first time the TimerTask is triggered.

<aquilesConfiguration>

<clusters>
<add friendlyName="Service">
<connection poolType="SIZECONTROLLEDPOOL" factoryType="BUFFERED" />
<endpointManager type="ROUNDROBIN" defaultTimeout="6000">
<cassandraEndpoints>
<add address="RT3" port="9160"/>
</cassandraEndpoints>
</endpointManager>
</add>
</clusters>
</aquilesConfiguration>

Reopened Issue: DeleteCommand validation exception when column value is null [11456]

javiercanillas — Fri, 14 Jun 2013 07:16:10 GMT

When using a DeleteCommand the input column must contain a not null value even though it is not actually needed for the deletation operation.

Reopened Issue: BatchMutateCommand with SuperColumns over a Standard ColumnFamily [11744]

javiercanillas — Fri, 14 Jun 2013 07:16:10 GMT

BatchMutateCommand does not validate the ColumnFamily Type before proceeding with the execution. Doing so, you can try to push an insert with a SuperColumn over a Standard SuperColumn.

This is also enable when using BatchMutateCommand with DeletationMutation.

Suppose the following code:

Guid id = Guid.NewGuid();
BatchMutateCommand insertCMD = new BatchMutateCommand();
insertCMD.KeySpace = "Keyspace1";
Dictionary<string, Dictionary<string, List<IAquilesMutation>>> keyMutations = new Dictionary<string, Dictionary<string, List<IAquilesMutation>>>();
Dictionary<string, List<IAquilesMutation>> columnMutation = new Dictionary<string, List<IAquilesMutation>>();
List<IAquilesMutation> mutations = new List<IAquilesMutation>();

AquilesSetMutation insertContextDataMutation = new AquilesSetMutation();
insertContextDataMutation.SuperColumn = new AquilesSuperColumn()
{
Name = "Data",
Columns = new List<AquilesColumn>()
};

insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), DateTime.UtcNow));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), DateTime.UtcNow));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), 0));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), false));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), 0));

mutations.Add(insertContextDataMutation);

columnMutation.Add("Standard1", mutations);
string key = id.ToString();
keyMutations.Add(key, columnMutation);
insertCMD.Mutations = keyMutations;

using (IAquilesConnection connection = AquilesHelper.RetrieveConnection("TestCluster1"))
{
connection.Execute(insertCMD);
}

Reopened Issue: MultiGetSliceCommand without SuperColumn over a Super ColumnFamily Type [11745]

javiercanillas — Fri, 14 Jun 2013 07:16:10 GMT

If you tried to execute a MultiGetSliceCommand without setting the SuperColumn (because you want to get all the child SuperColumns) over a ColumnFamily that is Super, you will get an exception with the message: "SuperColumn cannot be null or empty when accesing a Super type ColumnFamily.".

Suppose the following code:

Guid id = Guid.NewGuid();
BatchMutateCommand insertCMD = new BatchMutateCommand();
insertCMD.KeySpace = "Keyspace1";
Dictionary<string, Dictionary<string, List<IAquilesMutation>>> keyMutations = new Dictionary<string, Dictionary<string, List<IAquilesMutation>>>();
Dictionary<string,List<IAquilesMutation>> columnMutation = new Dictionary<string,List<IAquilesMutation>>();
List<IAquilesMutation> mutations = new List<IAquilesMutation>();

AquilesSetMutation insertContextDataMutation = new AquilesSetMutation();
insertContextDataMutation.SuperColumn = new AquilesSuperColumn()
{
Name = "Data",
Columns = new List<AquilesColumn>()
};

insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), DateTime.UtcNow));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), DateTime.UtcNow));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), 0));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), false));
insertContextDataMutation.SuperColumn.Columns.Add(this.CreateColumn(DateTime.Now.ToUniversalTime().Ticks.ToString(), 0));

mutations.Add(insertContextDataMutation);

columnMutation.Add("Super1", mutations);
string key = id.ToString();
keyMutations.Add(key, columnMutation);
insertCMD.Mutations = keyMutations;

MultiGetSliceCommand multiGetSliceCMD = new MultiGetSliceCommand() { KeySpace = "Keyspace1", ColumnFamily = "Super1" };

AquilesSliceRange range = new AquilesSliceRange();
range.FinishColumn = DateTime.Now.ToUniversalTime().Ticks.ToString();
range.Reversed = true;
range.Count = int.MaxValue;

AquilesSlicePredicate predicate = new AquilesSlicePredicate();
predicate.SliceRange = range;

multiGetSliceCMD.Keys = new List<string>() { key };
multiGetSliceCMD.Predicate = predicate;

using (IAquilesConnection connection = AquilesHelper.RetrieveConnection("TestCluster1"))
{
connection.Open();
connection.Execute(insertCMD);
connection.Execute(multiGetSliceCMD);
connection.Close();
}

Assert.IsNotNull(multiGetSliceCMD.Output);

Reopened Issue: GuidGenerator generates same Guid if called fast [12557]

apack12 — Fri, 14 Jun 2013 07:16:10 GMT

I try to generate multiple timeuuid using GuidGenerator.cs, but looks like it can generate duplicate uuid values sometimes.

Here is my code:

for (int i = 0; i < 10; i++)
{
Guid guid = Aquiles.Helpers.GuidGenerator.GenerateTimeBasedGuid();
Console.WriteLine(guid.ToString());
}

Here is the result:

7fda8f95-f444-11df-af7e-2cbfe486a130
7fdaddb6-f444-11df-af7e-2cbfe486a130
7fdaddb6-f444-11df-af7e-2cbfe486a130
7fdaddb6-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130
7fdb04c7-f444-11df-af7e-2cbfe486a130

Is it a problem? Some of my CFs are using TimeUUIDType, if there are some insert actions at the same time, the uuid will be the same. Any solutions? Thanks.

Posted by: xamlguide on discussions

Reopened Issue: GetKeyRangeSliceCommand doesn't support for empty Start and End Key [12796]

javiercanillas — Fri, 14 Jun 2013 07:16:10 GMT

To get all the entries in a columnfamily, cassandra expects the first startkey value to have StartKey.Length == 0. Current code doesn't allow this need something like the following:

1. Change AquilesKeyRange.cs ValidateNullOrEmptyStartKey to something like

private void ValidateNullOrEmptyStartKey()
{
// StartKey length == 0 means get the first available item
if (this.StartKey == null)
{
throw new AquilesCommandParameterException("StartKey must not be null.");
}
}

2. Also need to make an empty EndKey allowable - so something like

private void ValidateNullOrEmptyEndKey()
{
// EndKey can be null - means get as many as allowed by Count
}

3. Also need a type converter for IAquilesKeyTokenRange implementations - in ModelConverterHelper.cs around line 78 something like:

//Aquiles IAquilesKeyTokenRange converter
converter = new AquilesTokenRangeConverter();
converters.Add(typeof(AquilesTokenRange), converter);
converters.Add(typeof(AquilesKeyRange), converter);
converters.Add(typeof(KeyRange), converter);

4. Plus an implementation of the converter - something like:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using Aquiles.Model;
using Apache.Cassandra;

namespace Aquiles.Converter.Model.Impl
{
/// <summary>
/// Converts from Aquiles.Model.IAquilesKeyTokenRange (Aquiles.Model.AquilesTokenRange) to Apache.Cassandra.KeyRange
///
/// </summary>
class AquilesTokenRangeConverter : IThriftConverter<IAquilesKeyTokenRange, KeyRange>
{
#region IThriftConverter<IAquilesKeyTokenRange,KeyRange> Members

public KeyRange Transform(IAquilesKeyTokenRange objectA)
{
if (objectA is AquilesTokenRange)
{
AquilesTokenRange tokenRange = (AquilesTokenRange)objectA;
return tokenRange.ToThrift();
//KeyRange result = new KeyRange
//{
// Count = tokenRange.Count,
// Start_token = tokenRange.StartToken,
// End_token = tokenRange.EndToken
//};
}
else if (objectA is AquilesKeyRange)
{
AquilesKeyRange keyRange = (AquilesKeyRange)objectA;
return keyRange.ToThrift();
//KeyRange result = new KeyRange
//{
// Count = keyRange.Count,
// Start_key = keyRange.StartKey,
// End_key = keyRange.EndKey
//};
}
else
throw new ArgumentException(string.Format("Don't know how to convert {0}", objectA));
}

public IAquilesKeyTokenRange Transform(KeyRange objectB)
{
if (objectB.End_token != null || objectB.Start_token != null)
{
return new AquilesTokenRange
{
StartToken = objectB.Start_token,
EndToken = objectB.End_token,
Count = objectB.Count
};
}
else
{
return new AquilesKeyRange
{
StartKey = objectB.Start_key,
EndKey = objectB.End_key,
Count = objectB.Count
};
}
}

#endregion
}
}

found by: patrickthompson
Discussion: http://aquiles.codeplex.com/Thread/View.aspx?ThreadId=240791

Reopened Issue: UnavailableException and TimedOutException is not meaning of connection broken [12869]

javiercanillas — Fri, 14 Jun 2013 07:16:09 GMT

By reading again the Thrift Documentation, it seems that the UnavailableException and TimedOutException doesn't mean that the node you are connected is down or not healthy, so invaliding the connection and banning the endpoint is not a very clever idea.

I'm copying the part of the documention associated with it:

*UnavailableException*
Not all the replicas required could be created and/or read.
*TimedOutException*
The node responsible for the write or read did not respond during the rpc interval specified in your configuration (default 10s). This can happen if the request is too large, the node is oversaturated with requests, or the node is down but the failure detector has not yet realized it (usually this takes < 30s).

Closed Issue: divide by zero in SizeControlledConnectionPool.controlIdleClientSizeMethod [11340]

javiercanillas — Thu, 16 May 2013 10:32:03 GMT

Closed Issue: DeleteCommand validation exception when column value is null [11456]

javiercanillas — Thu, 16 May 2013 10:32:03 GMT

When using a DeleteCommand the input column must contain a not null value even though it is not actually needed for the deletation operation.

Closed Issue: BatchMutateCommand with SuperColumns over a Standard ColumnFamily [11744]

javiercanillas — Thu, 16 May 2013 10:32:03 GMT

Closed Issue: MultiGetSliceCommand without SuperColumn over a Super ColumnFamily Type [11745]

javiercanillas — Thu, 16 May 2013 10:32:02 GMT

Closed Issue: GuidGenerator generates same Guid if called fast [12557]

apack12 — Thu, 16 May 2013 10:32:02 GMT

Closed Issue: GetKeyRangeSliceCommand doesn't support for empty Start and End Key [12796]

javiercanillas — Thu, 16 May 2013 10:32:02 GMT

Closed Issue: UnavailableException and TimedOutException is not meaning of connection broken [12869]

javiercanillas — Thu, 16 May 2013 10:32:01 GMT

New Post: Socked Exception When Mutate

JamesYYang — Wed, 16 Jan 2013 04:42:31 GMT

We get a socket exception when mutate value:

[SocketException (0x274c): A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond]
   System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags) +6210712
   System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) +134

[IOException: Unable to read data from the transport connection: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.]
   System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size) +300
   Thrift.Transport.TStreamTransport.Read(Byte[] buf, Int32 off, Int32 len) +141
   Thrift.Transport.TTransport.ReadAll(Byte[] buf, Int32 off, Int32 len) +90
   Thrift.Transport.TFramedTransport.ReadFrame() +88
   Thrift.Transport.TFramedTransport.Read(Byte[] buf, Int32 off, Int32 len) +143
   Thrift.Transport.TTransport.ReadAll(Byte[] buf, Int32 off, Int32 len) +90
   Thrift.Protocol.TBinaryProtocol.ReadAll(Byte[] buf, Int32 off, Int32 len) +73
   Thrift.Protocol.TBinaryProtocol.ReadI32() +53
   Thrift.Protocol.TBinaryProtocol.ReadMessageBegin() +79
   Apache.Cassandra.Client.recv_batch_mutate() +34
   Apache.Cassandra.Client.batch_mutate(Dictionary`2 mutation_map, ConsistencyLevel consistency_level) +24

But query is fine.

This is exception happened today, but in past month it worked fine.

Any idea?

New Post: Anyway to reset TTL for a column

paiwan — Thu, 29 Nov 2012 16:21:11 GMT

You just need to overwrite(insert) new column with the same name and new TTL.

New Post: Anyway to reset TTL for a column

dennisleng — Wed, 21 Nov 2012 02:07:28 GMT

Hi All

Just wondering anyway to reset TTL for a column as it could work the same way as asp.net sliding expiration?

Cheers

New Post: Exception 'Unable to read data from the transport connection:

vegah — Wed, 31 Oct 2012 07:55:53 GMT

I don't think I understand the DefaultClient.cs class. There is a do-while loop in the execute method, that retries MaximumRetries times - but as far as I can see, MaximumRetries is always set to 0, which means that it will never try more than one node. I though the idea was that if a node was down, it would try another one from the pool. I am probably misunderstanding something, because also if an exception is not marked as "ShouldRetry" the executioncounter is reset to 0 - which means that it will start from 0 again? What am I misunderstanding?

New Post: Exception 'Unable to read data from the transport connection:

vegah — Tue, 30 Oct 2012 13:09:13 GMT

I from time to time get this error when using the dev-webserver (included in vs) after a time of inactivity (hours, days). It does not seem to be able to recover without restarting the dev webserver.

Isn't this also a case where Aquiles should retry the connection or switch node? In DefaultCluster.cs I find a "catch (ExecutionBlockException ex)", but DefaultClient.buildException does not handle this exception. Not that I think it would matter, because as I said it does not seem to be able to recover without a restart. Any suggestions?

New Post: CompositeType Support

vegah — Tue, 30 Oct 2012 12:57:17 GMT

Added a patch that adds support for composite columns through a helper class. I have tested it, and it seems to work.