v1.0.2

Merge 1.0.2 into master

Author: Abductcows

README.md

@@ -7,36 +7,38 @@
 ### TL;DR: Faster List<Boolean> than ArrayList with the same exact behaviour. 
 - [Documentation](https://abductcows.github.io/java-bit-array/gr/geompokon/bitarray/BitArray.html) 
 - [Release](https://github.com/Abductcows/java-bit-array/releases/latest)
-
+- [Benchmarks](https://github.com/Abductcows/bit-array-benchmarks)
 ### Motivation
 This class is a replacement for the `ArrayList<Boolean>` type when working with its `List` interface. It boasts higher performance in add, remove and set operations and requires less memory for storing the same elements. 
 
 The BitArray is by all means an array; it is random access and all elements have contiguous indices at all times. Its behaviour is meant to be indistinguishable from ArrayList in order for programmers to be able to substitute it for the latter and benefit from its performance advantage. 
 
 ### Few details
-Internally the array stores the boolean elements in an array of long primitives. These long primitives essentially form a sequence of bits; their chained bit representation in 2's complement. Boolean elements are converted to and from their bit equivalent to perform insertions, deletions etc. With the appropriate bitwise operations new elements can be added at a specific index and elements already in the array can be shifted to preserve the previous order. Thanks to that hack, element shifting and array resizing is much cheaper, all while the elements themselves occupy less space in memory.
+Internally the array stores the boolean elements in an array of long primitives. These long primitives essentially form a sequence of bits; their chained bit representation in 2's complement. Boolean elements are converted to and from their bit equivalent to perform insertions, deletions etc. With the appropriate bitwise operations new elements can be added at a specific index and elements already in the array can be shifted to preserve the previous order. Thanks to this "hack", element shifting and array resizing is much cheaper, all while the elements themselves occupy less space in memory.
 
-### Performance
-With regard to the difference in performance, I have included a [temporary benchmark](https://github.com/Abductcows/java-bit-array/blob/dev/src/test/java/gr/geompokon/bitarray/BitArrayVsArrayListBenchmarkTest.java) file for you to test. A conservative report on my findings based on that is that this array is about 4-5 times faster in random index `add` and `remove` operations. `get` and `set` run at about the same time. I am looking into creating a more trustworthy benchmark using a benchmark framework like [JMH](https://github.com/openjdk/jmh) in order to be able to publish some results with confidence. If you have experience doing that and want to contribute, feel free to start an [issue](https://github.com/Abductcows/java-bit-array/issues).
+### Thread safety
+The class is not currently thread-safe, I will look into it properly at some point. For the time being you can use `Collections.synchronizedList()`
 
-### Disclaimer
-As you can tell from the project version and creation date, this class is very new and not battle-tested. As such I would discourage you from using it in a serious application just yet.
+### Null values
+Unfortunately, due to the implementation I have not been able to accommodate null values in the array. Null insertions or updates will throw NullPointerException. 
+
+### Performance
+For the performance difference, check out the [benchmark repository](https://github.com/Abductcows/bit-array-benchmarks). It includes results from my runs and the benchmark files should you want to run them yourself. A TLDR version is that it gets much faster the more the elements are in add/remove. The performance difference stems wholly from resizes and moves. For example an insertion at random indices of 1000 elements with an initial capacity of 10 runs at 2x the speed. Same scenario but for 1.5M elements and the BitArray runs 13x faster. But for already resized arrays and insertions at the tail, the difference is miniscule. The numbers mentioned are quite conservative for safety. Also, it can easily handle `INTEGER.MAX_VALUE` elements, but cannot hold more. 
 
 # Getting Started
-You will need the class and source files. You can grab the [latest release](https://github.com/Abductcows/java-bit-array/releases/latest) (built with jdk-11) or download the project and run `mvn package/install` yourself. Releases contain a zip file with separate jars for classes, sources and javadoc. Include at least the class jar in your project and you will be able to use the BitArray. Looks like you are good to go.
+You will need the class and source files. You can grab the [latest release](https://github.com/Abductcows/java-bit-array/releases/latest) (built with jdk-11) or download the project and run `mvn install` yourself. Releases contain a zip file with separate jars for classes, sources and javadoc. Include at least the class jar in your project and you will be able to use the BitArray. Looks like you are good to go.
 
 ### Versioning
 The project uses [SemVer](https://semver.org/) for versioning.
 
 ### Contributions and future of this project
 I would like to work on this project with anyone willing to contribute. My estimation of the rough priority of actions needed is:
 
-- Testing/debugging: Write better and well documented tests to enhance confidence
-- Benchmarking: Give ArrayList a run for their money
+- Testing: Improve tests to enhance confidence
 - Optimizing: 'cause why not. Maybe override a few of the AbstractList's default implementations
-- New features: Not sure what to add, suggestions very welcome
+- New features: Not sure if there is anything to add, suggestions very welcome
 
-If you want to contribute, check out [CONTRIBUTING.md](https://github.com/Abductcows/java-bit-array/blob/master/CONTRIBUTING.md) for more info.
+I would also appreciate you sharing your opinion on this class and the project as a whole. If you want to contribute, check out [CONTRIBUTING.md](https://github.com/Abductcows/java-bit-array/blob/master/CONTRIBUTING.md) for more info.
 
 ### License
 This Project is licensed under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0)

pom.xml

@@ -7,7 +7,7 @@
 
     <groupId>gr.geompokon</groupId>
     <artifactId>bit-array</artifactId>
-    <version>1.0.0</version>
+    <version>1.0.2</version>
 
     <name>BitArray</name>
     <description>Java List of Booleans class with reduced memory usage and higher performance</description>
@@ -25,6 +25,14 @@
             <groupId>org.junit.jupiter</groupId>
             <artifactId>junit-jupiter-engine</artifactId>
             <version>5.1.0</version>
+            <scope>test</scope>
+            <optional>true</optional>
+        </dependency>
+        <dependency>
+            <groupId>org.junit.jupiter</groupId>
+            <artifactId>junit-jupiter-params</artifactId>
+            <version>5.1.0</version>
+            <scope>test</scope>
             <optional>true</optional>
         </dependency>
     </dependencies>

src/main/java/gr/geompokon/bitarray/BitArray.java

@@ -50,7 +50,7 @@
  * not follow the one bit per entry principle.
  * </p>
  *
- * @version 1.0.0
+ * @version 1.0.2
  * @see java.util.List
  * @see java.util.AbstractList
  * @see java.util.ArrayList
@@ -198,7 +198,6 @@ public Boolean get(int index) {
     public Boolean set(int index, Boolean bit) {
         Objects.requireNonNull(bit);
         ensureIndexInRange(index, elements - 1);
-        modCount++;
         // get bit indices
         int longIndex = getLongIndex(index);
         int indexInLong = getIndexInLong(index);
@@ -319,43 +318,41 @@ private void addAndShiftAllRight(boolean bit, int longIndex, int indexInLong) {
         int maxLongIndex = getLongIndex(elements);
         // add the bit and save the LSB that was shifted out
         int bitIntValue = Boolean.compare(bit, Boolean.FALSE);
-        int rightmostBit = insertInLong(bitIntValue, longIndex++, indexInLong);
+        long rightmostBit = insertInLong(bitIntValue, 1, longIndex++, indexInLong);
         // keep inserting old LSB at 0 of next long and moving on with the new LSB
         while (longIndex <= maxLongIndex) {
-            rightmostBit = insertInLong(rightmostBit, longIndex++, 0);
+            rightmostBit = insertInLong(rightmostBit, 1, longIndex++, 0);
         }
     }
 
     /**
-     * Inserts the bit in the index of the long specified by the arguments and returns the previous LSB.
-     *
-     * <p>
-     * Inserting at any index is done by splitting the long word in two parts and rejoining them after shifting and
-     * setting the new bit. The LSB that is shifted out is returned.
-     * </p>
+     * Inserts the {@code lastLength} rightmost bits of lastValue in the position specified by {@code longIndex} and
+     * {@code indexInLong}, and then shifts every element with index >= {@code indexInLong} to the right. The bits that
+     * are shifted out are returned in the leftmost position
      *
-     * @param bit         the bit to be inserted
+     * @param lastValue   bits to be inserted into the long
+     * @param lastLength  length in bits of the last value
      * @param longIndex   index of the long in the {@code data} array
-     * @param indexInLong index of the bit in the long
-     * @return LSB of the long before insertion
+     * @param indexInLong index of the insertion bit in the long
+     * @return bits that were shifted out due to the insertion
      */
-    private int insertInLong(int bit, int longIndex, int indexInLong) {
-        // get right side [indexInLong : ], can not be empty, will be shifted
+    private long insertInLong(long lastValue, int lastLength, int longIndex, int indexInLong) {
+        // select the bits [indexInLong, (word end)] for the insertion
         long rightSide = (data[longIndex] << indexInLong) >>> indexInLong;
-        // get left side [0 : indexInLong), can be empty, will remain intact
-        long leftSide = data[longIndex] - rightSide;
-
-        // save LSB
-        long rightSideLSB = rightSide & 1L;
-        // unsigned shift to the right to make space for the new bit
-        rightSide >>>= 1;
-        // set the new bit
-        rightSide |= (long) bit << (BITS_PER_LONG - 1 - indexInLong);
+        // separate the left part, this will remain intact
+        long leftSide = data[longIndex] & ~rightSide;
+
+        // save the bits that will be shifted out
+        long rightSideShiftOut = selectBits(rightSide, BITS_PER_LONG - lastLength, lastLength);
+        // unsigned shift to the right to make space for the new bits
+        rightSide >>>= lastLength;
+        // set the new bits
+        rightSide |= lastValue << (BITS_PER_LONG - lastLength - indexInLong);
         // re-join the two parts
-        data[longIndex] = leftSide + rightSide;
+        data[longIndex] = leftSide ^ rightSide;
 
-        // return the LSB
-        return (int) rightSideLSB;
+        // return the discarded bits
+        return rightSideShiftOut;
     }
 
     /**
@@ -367,47 +364,63 @@ private int insertInLong(int bit, int longIndex, int indexInLong) {
     private void removeAndShiftAllLeft(int longIndex, int indexInLong) {
         // start at the end and work back to current long index
         int currentLongIndex = getLongIndex(elements - 1);
-        int leftmostBit = 0; // dud value for first shift
+        long leftmostBit = 0; // dud value for first shift
         // keep adding the old MSB as LSB of the previous long index and shifting the rest to the left
         while (currentLongIndex > longIndex) {
-            leftmostBit = appendBitAndRemoveAtIndex(leftmostBit, currentLongIndex--, 0);
+            leftmostBit = removeAtIndexAndAppend(leftmostBit, 1, currentLongIndex--, 0);
         }
-        // add the final MSB as LSB of {@code longIndex} and shift only the bits to the removed's right
-        appendBitAndRemoveAtIndex(leftmostBit, longIndex, indexInLong);
+        // add the final MSB as LSB of longIndex and shift only the bits to the popped bit's right
+        removeAtIndexAndAppend(leftmostBit, 1, longIndex, indexInLong);
     }
 
     /**
-     * Appends the bit at the end of the long specified by the arguments and removes the bit at {@code indexInLong}.
-     *
-     * <p>
-     * Since {@code indexInLong} can be at the middle of the long word, removing the bit is done by splitting the
-     * long in two parts, clearing the desired bit and shifting once to restore the order of the previous bits.
-     * </p>
+     * Removes the {@code lastLength} bits from the long specified by {@code longIndex} starting from {@code indexInLong}
+     * and then appends the same length of bits from {@code lastValue} at the end of the long. The
      *
-     * @param bit         the bit to be appended to the long
+     * @param lastValue   bits to be appended to the long
+     * @param lastLength  length in bits of the last value
      * @param longIndex   index of the long in the {@code data} array
-     * @param indexInLong index of the bit in the long
-     * @return bit at {@code longIndex} that was popped out
+     * @param indexInLong index of the first removed bit in the long
+     * @return bits that were popped from the long
      */
-    private int appendBitAndRemoveAtIndex(int bit, int longIndex, int indexInLong) {
+    private long removeAtIndexAndAppend(long lastValue, int lastLength, int longIndex, int indexInLong) {
         // get right side [indexInLong : ], can not be empty, will be shifted
         long rightSide = (data[longIndex] << indexInLong) >>> indexInLong;
         // get left side [0 : indexInLong), can be empty, will remain intact
-        long leftSide = data[longIndex] - rightSide;
+        long leftSide = data[longIndex] & ~rightSide;
+
+        // save removed values
+        long poppedValues = selectBits(rightSide, indexInLong, lastLength) >>> (BITS_PER_LONG - indexInLong - lastLength);
 
-        // save MSB
-        int rightSideMSB = getBitInLong(rightSide, indexInLong);
-        // clear MSB and shift to the left to make it "disappear"
-        rightSide &= ~singleBitMask(indexInLong);
-        rightSide <<= 1;
-        // append the previous bit
-        rightSide += bit;
+        // clear copied bits and shift to the left
+        rightSide = (rightSide << indexInLong + lastLength) >>> indexInLong;
+        // append the previous bits
+        rightSide |= lastValue;
 
         // re-join the two parts
-        data[longIndex] = leftSide + rightSide;
+        data[longIndex] = leftSide ^ rightSide;
 
-        // return the MSB
-        return rightSideMSB;
+        // return the popped bits
+        return poppedValues;
+    }
+
+    /**
+     * Returns a long bit mask with ones only in the range [start, start + length)
+     *
+     * @param start  start index of the selection
+     * @param length number of set bits in the result
+     * @return bit mask covering the range specified
+     * @implSpec <p>
+     * {@code start} should be in the range [0, 63]<br>
+     * {@code length} should be in the range [1, 64]<br>
+     * {@code start} and {@code length} should satisfy: start + length <= {@link #BITS_PER_LONG}
+     * </p>
+     */
+    private long selectBits(long aLong, int start, int length) {
+        long mask = Long.MIN_VALUE >>> start; // need at least the first bit
+        mask |= (Long.MIN_VALUE >>> start) - 1; // make everything to the right ones
+        mask &= -(Long.MIN_VALUE >>> (start + length - 1)); // make everything from end of length and forward 0
+        return aLong & mask;
     }
 
     /**
@@ -430,6 +443,11 @@ private void ensureIndexInRange(int index, int endInclusive) {
      * </p>
      */
     private void ensureCapacity() {
+        // check for completely full array
+        if (elements == Integer.MAX_VALUE) {
+            throw new IllegalStateException("Cannot insert; array is completely full. Size = " + size());
+        }
+        // extend if currently full
         if (elements == data.length * BITS_PER_LONG) {
             doubleSize();
         }
@@ -439,7 +457,10 @@ private void ensureCapacity() {
      * Doubles the size of the array.
      */
     private void doubleSize() {
-        resize(2 * elements);
+        // make sure new element count does not overflow
+        // we can't index more than Integer.MAX_VALUE elements through the List interface anyway
+        int newSize = (int) Math.min(2L * elements, Integer.MAX_VALUE);
+        resize(newSize);
     }
 
     /**
@@ -496,7 +517,6 @@ private int longsRequiredForNBits(int nBits) {
                 (double) nBits / BITS_PER_LONG);
     }
 
-
     /*
         BitArray specific methods
     */
@@ -557,17 +577,14 @@ public String toString() {
      */
     public static BitArray fromString(String stringArray) {
 
-        String start = "Size = ";
+        final String start = "Size = ";
 
         if (!stringArray.startsWith(start)) {
             throw new UnknownFormatConversionException("Not a valid BitArray string");
         }
 
         // count number of digits of the array size
-        int currentIndex = start.length();
-        while (stringArray.charAt(currentIndex) != ',') {
-            currentIndex++;
-        }
+        int currentIndex = stringArray.indexOf(",", start.length());
 
         int arraySize;
         try {