Skip to content
This repository has been archived by the owner on Aug 30, 2022. It is now read-only.

Project renaming XP-225 #165

Merged
merged 7 commits into from
Dec 2, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 6 additions & 6 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: install dependencies
Expand All @@ -25,7 +25,7 @@ jobs:
- save_cache:
paths:
- ./venv
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: run tests
Expand All @@ -43,7 +43,7 @@ jobs:
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: install dependencies
Expand All @@ -55,7 +55,7 @@ jobs:
- save_cache:
paths:
- ./venv
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: run tests slow
Expand All @@ -73,7 +73,7 @@ jobs:
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: install dependencies
Expand All @@ -85,7 +85,7 @@ jobs:
- save_cache:
paths:
- ./venv
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: run tests benchmark
Expand Down
9 changes: 4 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a
Expand All @@ -17,7 +18,6 @@ For reference, the possible headings are:
- `External Contributors` to list all external contributors.
- `Notes` for notes regarding this particular release.


## [Unreleased]

## [0.1.0] - 2019-09-25
Expand All @@ -27,9 +27,8 @@ The first public release of **XAIN**
### Added

- FedML implementation on well known
[benchmarks](https://github.com/xainag/xain/tree/master/benchmarks/benchmark) using
[benchmarks](https://github.com/xainag/xain-fl/tree/master/benchmarks/benchmark) using
a realistic deep learning model structure.


[Unreleased]: https://github.com/xainag/xain/pulls?utf8=%E2%9C%93&q=merged%3A%3E2019-09-25+
[0.1.0]: https://github.com/xainag/xain/pulls?utf8=%E2%9C%93&q=merged%3A%3C%3D2019-09-25+
[unreleased]: https://github.com/xainag/xain-fl/pulls?utf8=%E2%9C%93&q=merged%3A%3E2019-09-25+
[0.1.0]: https://github.com/xainag/xain-fl/pulls?utf8=%E2%9C%93&q=merged%3A%3C%3D2019-09-25+
10 changes: 5 additions & 5 deletions CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
/scripts/ @tanertopal @danieljanes

## Python packages
/xain/datasets/ @tanertopal
/xain/fl/ @danieljanes
/xain/grpc/ @r-marques @finiteprods
/xain/helpers/ @danieljanes @tanertopal
/xain/types/ @danieljanes @tanertopal
/xain_fl/datasets/ @tanertopal
/xain_fl/fl/ @danieljanes
/xain_fl/grpc/ @r-marques @finiteprods
/xain_fl/helpers/ @danieljanes @tanertopal
/xain_fl/types/ @danieljanes @tanertopal
8 changes: 4 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ RUN python -m pip install -U pip==19.3.1 setuptools==41.6.0
COPY setup.py setup.py

# These files are needed for the setup.py to work
COPY xain/__version__.py xain/__version__.py
COPY xain_fl/__version__.py xain_fl/__version__.py
COPY README.md README.md

# Install only install_requires
RUN python setup.py egg_info && \
LN=$(awk '/tensorflow/{ print NR; exit }' xain.egg-info/requires.txt) && \
IR=$(head -n $LN xain.egg-info/requires.txt | awk '{gsub(/\[.+\]/,"");}1') && \
LN=$(awk '/tensorflow/{ print NR; exit }' xain_fl.egg-info/requires.txt) && \
IR=$(head -n $LN xain_fl.egg-info/requires.txt | awk '{gsub(/\[.+\]/,"");}1') && \
python -m pip install $IR

COPY xain xain
COPY xain_fl xain_fl
COPY protobuf protobuf

RUN python -m pip install .
24 changes: 13 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[![CircleCI](https://img.shields.io/circleci/build/github/xainag/xain/master?style=flat-square)](https://circleci.com/gh/xainag/xain/tree/master)
[![PyPI](https://img.shields.io/pypi/v/xain?style=flat-square)](https://pypi.org/project/xain/)
[![GitHub license](https://img.shields.io/github/license/xainag/xain?style=flat-square)](https://github.com/xainag/xain/blob/master/LICENSE)
[![CircleCI](https://img.shields.io/circleci/build/github/xainag/xain-fl/master?style=flat-square)](https://circleci.com/gh/xainag/xain-fl/tree/master)
[![PyPI](https://img.shields.io/pypi/v/xain-fl?style=flat-square)](https://pypi.org/project/xain-fl/)
[![GitHub license](https://img.shields.io/github/license/xainag/xain-fl?style=flat-square)](https://github.com/xainag/xain-fl/blob/master/LICENSE)

# XAIN

Expand All @@ -14,29 +14,30 @@ POLITE NOTE: We want to point out that running the benchmarks as described below

## Quick Start

XAIN requires [Python 3.6+](https://python.org/). To install the `xain` package just run:
XAIN requires [Python 3.6+](https://python.org/). To install the `xain-fl` package just run:

```shell
$ python -m pip install xain
$ python -m pip install xain-fl
```

XAIN can also be installed with GPU support through the `gpu` extra feature. To
install the `xain` package with support for GPUs just run:
install the `xain-fl` package with support for GPUs just run:

```shell
$ python -m pip install xain[gpu]
$ python -m pip install xain-fl[gpu]
```

### Running training sessions and benchmarks

To run training sessions, see the [benchmark
package](https://github.com/xainag/xain/tree/master/benchmarks/benchmark) and the
package](https://github.com/xainag/xain-fl/tree/master/benchmarks/benchmark) and the
[benchmark
documentation](https://github.com/xainag/xain/blob/master/docs/quick.md#training).
documentation](https://github.com/xainag/xain-fl/blob/master/docs/quick.md#training).

## Install from source

For development we require some extra system dependencies:

- [clang-format 8+](https://clang.llvm.org/docs/ClangFormat.html)
- Linux: `sudo apt install clang-format`
- macOS: `brew install clang-format`
Expand All @@ -46,8 +47,8 @@ For development we require some extra system dependencies:
To clone this repository and to install the XAIN project, please execute the following commands:

```shell
$ git clone https://github.com/xainag/xain.git
$ cd xain
$ git clone https://github.com/xainag/xain-fl.git
$ cd xain-fl

$ python -m pip install -e .[dev]
```
Expand All @@ -64,6 +65,7 @@ $ pytest

The project documentation resides under `docs/`. To build the documentation
run:

```shell
$ cd docs/
$ make docs
Expand Down
57 changes: 28 additions & 29 deletions RELEASE_PROCESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,26 +17,26 @@ A release on git is just a tagged commit on the `master` branch.
Here we detail the process of creating a new Github release.

1. Create and merge a pull request that:
- increases the version number in
[`xain/__version__.py`](https://github.com/xainag/xain/blob/master/xain/__version__.py)
according the versioning schema.
- updates the
[`CHANGELOG.md`](https://github.com/xainag/xain/blob/master/CHANGELOG.md)
with all notable changes for the release.
- possibly update the `Development Status` classifiers in the
[`setup.py`](https://github.com/xainag/xain/blob/master/setup.py). You
can check supported classifiers in the [pypi
website](https://pypi.org/classifiers/).
2. Got to the [Github Releases tab](https://github.com/xainag/xain/releases)
- increases the version number in
[`xain_fl/__version__.py`](https://github.com/xainag/xain-fl/blob/master/xain_fl/__version__.py)
according the versioning schema.
- updates the
[`CHANGELOG.md`](https://github.com/xainag/xain-fl/blob/master/CHANGELOG.md)
with all notable changes for the release.
- possibly update the `Development Status` classifiers in the
[`setup.py`](https://github.com/xainag/xain-fl/blob/master/setup.py). You
can check supported classifiers in the [pypi
website](https://pypi.org/classifiers/).
2. Got to the [Github Releases tab](https://github.com/xainag/xain-fl/releases)
and create a new release:
- for the tag version use the version defined in 1. preceded by a `v`, e.g.
v0.3.2, and target master.
- for the release title use the same as the tag version.
- for the release description, copy the section from the
[`CHANGELOG.md`](https://github.com/xainag/xain/blob/master/CHANGELOG.md)
related to this version.
- possibly check the `This is a pre-release` check box.
- Publish the release.
- for the tag version use the version defined in 1. preceded by a `v`, e.g.
v0.3.2, and target master.
- for the release title use the same as the tag version.
- for the release description, copy the section from the
[`CHANGELOG.md`](https://github.com/xainag/xain-fl/blob/master/CHANGELOG.md)
related to this version.
- possibly check the `This is a pre-release` check box.
- Publish the release.

### How to publish a new release to PyPi

Expand All @@ -45,15 +45,14 @@ You can check more information in the [Python Packaging User
Guide](https://packaging.python.org/tutorials/packaging-projects/).

1. Checkout the current git tag e.g.
```bash
$ git checkout v0.3.2
```
```bash
$ git checkout v0.3.2
```
2. Generate the distribution archives:
```bash
$ python setup.py sdist bdist_wheel
```
```bash
$ python setup.py sdist bdist_wheel
```
3. Upload the distribution archives using the correct PyPi credentials:
```bash
$ python -m twine upload dist/*
```

```bash
$ python -m twine upload dist/*
```
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/aggregation.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
task_accuracies,
)
from benchmarks.helpers import storage
from xain.logger import get_logger
from xain_fl.logger import get_logger

FLAGS = flags.FLAGS

Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/final_task_accuracies.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from absl import flags, logging

from benchmarks.helpers import storage
from xain.types import PlotValues, XticksLabels, XticksLocations
from xain_fl.types import PlotValues, XticksLabels, XticksLocations

from .plot import plot
from .results import GroupResult, TaskResult
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import pytest
from absl import flags

from xain.helpers import sha1
from xain_fl.helpers import sha1

from . import final_task_accuracies
from .results import TaskResult
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/learning_rate.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from absl import flags, logging

from benchmarks.helpers import storage
from xain.types import PlotValues
from xain_fl.types import PlotValues

from .plot import plot
from .results import GroupResult, TaskResult
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/benchmark/aggregation/learning_rate_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
import pytest
from absl import flags

from xain.helpers import sha1
from xain.logger import get_logger
from xain_fl.helpers import sha1
from xain_fl.logger import get_logger

from . import learning_rate

Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/participant_hist.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from benchmarks.benchmark.aggregation.plot import plot_history_data
from benchmarks.benchmark.aggregation.results import GroupResult, TaskResult
from benchmarks.helpers.storage import create_output_subdir, fname_with_default_dir
from xain.types import Metrics
from xain_fl.types import Metrics

FLAGS = flags.FLAGS

Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/plot.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from numpy import ndarray

from benchmarks.helpers import storage
from xain.types import PlotValues
from xain_fl.types import PlotValues

matplotlib.use("AGG")

Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/results.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from typing import List, Optional, cast

from benchmarks.helpers import storage
from xain.types import Metrics
from xain_fl.types import Metrics


class TaskResult(ABC):
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/task_accuracies.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from absl import flags, logging

from benchmarks.helpers import storage
from xain.types import PlotValues
from xain_fl.types import PlotValues

from .plot import plot
from .results import GroupResult, TaskResult
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/task_accuracies_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

import pytest

from xain.helpers import sha1
from xain_fl.helpers import sha1

from . import task_accuracies

Expand Down
6 changes: 3 additions & 3 deletions benchmarks/benchmark/bench_ea.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
from benchmarks.benchmark.exec import run
from benchmarks.benchmark.net import orig_cnn_compiled
from benchmarks.helpers import storage
from xain.datasets import load_splits
from xain.fl.coordinator.aggregate import EvoAgg
from xain.fl.coordinator.evaluator import Evaluator
from xain_fl.datasets import load_splits
from xain_fl.fl.coordinator.aggregate import EvoAgg
from xain_fl.fl.coordinator.evaluator import Evaluator

DEFAULT_R = 50
DEFAULT_E = 1 # Number of training epochs in each round
Expand Down
6 changes: 3 additions & 3 deletions benchmarks/benchmark/benchmark_test.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from benchmarks.ops.run import cores
from xain.datasets.dataset import config
from xain_fl.datasets.dataset import config

from .aggregation.aggregation import aggregations
from .benchmark import benchmarks
Expand Down Expand Up @@ -34,7 +34,7 @@ def test_valid_model_names():

def test_valid_dataset_names():
"""
Verify that all dataset names used in `Task` objects are available in `xain.datasets`
Verify that all dataset names used in `Task` objects are available in `xain_fl.datasets`
"""

# Prepare
Expand All @@ -50,7 +50,7 @@ def test_valid_dataset_names():

def test_valid_instance_cores():
"""
Verify that all dataset names used in `Task` objects are available in `xain.datasets`
Verify that all dataset names used in `Task` objects are available in `xain_fl.datasets`
"""

# Prepare
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/exec/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from benchmarks.helpers import storage
from benchmarks.ops import results
from xain.datasets import load_splits
from xain_fl.datasets import load_splits

from . import run

Expand Down
Loading