Skip to content
This repository has been archived by the owner on Aug 30, 2022. It is now read-only.

Commit

Permalink
Project renaming XP-225 (#165)
Browse files Browse the repository at this point in the history
* Rename package (part of XP-225)
  • Loading branch information
Taner Topal authored Dec 2, 2019
1 parent df54999 commit 481a9af
Show file tree
Hide file tree
Showing 189 changed files with 346 additions and 339 deletions.
12 changes: 6 additions & 6 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ jobs:
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: install dependencies
Expand All @@ -25,7 +25,7 @@ jobs:
- save_cache:
paths:
- ./venv
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: run tests
Expand All @@ -43,7 +43,7 @@ jobs:
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: install dependencies
Expand All @@ -55,7 +55,7 @@ jobs:
- save_cache:
paths:
- ./venv
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: run tests slow
Expand All @@ -73,7 +73,7 @@ jobs:
# Download and cache dependencies
- restore_cache:
keys:
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
- v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: install dependencies
Expand All @@ -85,7 +85,7 @@ jobs:
- save_cache:
paths:
- ./venv
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain/config.py" }}
key: v1-dependencies-{{ checksum "setup.py" }}-{{ checksum "xain_fl/config.py" }}

- run:
name: run tests benchmark
Expand Down
9 changes: 4 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a
Expand All @@ -17,7 +18,6 @@ For reference, the possible headings are:
- `External Contributors` to list all external contributors.
- `Notes` for notes regarding this particular release.


## [Unreleased]

## [0.1.0] - 2019-09-25
Expand All @@ -27,9 +27,8 @@ The first public release of **XAIN**
### Added

- FedML implementation on well known
[benchmarks](https://github.com/xainag/xain/tree/master/benchmarks/benchmark) using
[benchmarks](https://github.com/xainag/xain-fl/tree/master/benchmarks/benchmark) using
a realistic deep learning model structure.


[Unreleased]: https://github.com/xainag/xain/pulls?utf8=%E2%9C%93&q=merged%3A%3E2019-09-25+
[0.1.0]: https://github.com/xainag/xain/pulls?utf8=%E2%9C%93&q=merged%3A%3C%3D2019-09-25+
[unreleased]: https://github.com/xainag/xain-fl/pulls?utf8=%E2%9C%93&q=merged%3A%3E2019-09-25+
[0.1.0]: https://github.com/xainag/xain-fl/pulls?utf8=%E2%9C%93&q=merged%3A%3C%3D2019-09-25+
10 changes: 5 additions & 5 deletions CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,8 @@
/scripts/ @tanertopal @danieljanes

## Python packages
/xain/datasets/ @tanertopal
/xain/fl/ @danieljanes
/xain/grpc/ @r-marques @finiteprods
/xain/helpers/ @danieljanes @tanertopal
/xain/types/ @danieljanes @tanertopal
/xain_fl/datasets/ @tanertopal
/xain_fl/fl/ @danieljanes
/xain_fl/grpc/ @r-marques @finiteprods
/xain_fl/helpers/ @danieljanes @tanertopal
/xain_fl/types/ @danieljanes @tanertopal
8 changes: 4 additions & 4 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -13,16 +13,16 @@ RUN python -m pip install -U pip==19.3.1 setuptools==41.6.0
COPY setup.py setup.py

# These files are needed for the setup.py to work
COPY xain/__version__.py xain/__version__.py
COPY xain_fl/__version__.py xain_fl/__version__.py
COPY README.md README.md

# Install only install_requires
RUN python setup.py egg_info && \
LN=$(awk '/tensorflow/{ print NR; exit }' xain.egg-info/requires.txt) && \
IR=$(head -n $LN xain.egg-info/requires.txt | awk '{gsub(/\[.+\]/,"");}1') && \
LN=$(awk '/tensorflow/{ print NR; exit }' xain_fl.egg-info/requires.txt) && \
IR=$(head -n $LN xain_fl.egg-info/requires.txt | awk '{gsub(/\[.+\]/,"");}1') && \
python -m pip install $IR

COPY xain xain
COPY xain_fl xain_fl
COPY protobuf protobuf

RUN python -m pip install .
24 changes: 13 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[![CircleCI](https://img.shields.io/circleci/build/github/xainag/xain/master?style=flat-square)](https://circleci.com/gh/xainag/xain/tree/master)
[![PyPI](https://img.shields.io/pypi/v/xain?style=flat-square)](https://pypi.org/project/xain/)
[![GitHub license](https://img.shields.io/github/license/xainag/xain?style=flat-square)](https://github.com/xainag/xain/blob/master/LICENSE)
[![CircleCI](https://img.shields.io/circleci/build/github/xainag/xain-fl/master?style=flat-square)](https://circleci.com/gh/xainag/xain-fl/tree/master)
[![PyPI](https://img.shields.io/pypi/v/xain-fl?style=flat-square)](https://pypi.org/project/xain-fl/)
[![GitHub license](https://img.shields.io/github/license/xainag/xain-fl?style=flat-square)](https://github.com/xainag/xain-fl/blob/master/LICENSE)

# XAIN

Expand All @@ -14,29 +14,30 @@ POLITE NOTE: We want to point out that running the benchmarks as described below

## Quick Start

XAIN requires [Python 3.6+](https://python.org/). To install the `xain` package just run:
XAIN requires [Python 3.6+](https://python.org/). To install the `xain-fl` package just run:

```shell
$ python -m pip install xain
$ python -m pip install xain-fl
```

XAIN can also be installed with GPU support through the `gpu` extra feature. To
install the `xain` package with support for GPUs just run:
install the `xain-fl` package with support for GPUs just run:

```shell
$ python -m pip install xain[gpu]
$ python -m pip install xain-fl[gpu]
```

### Running training sessions and benchmarks

To run training sessions, see the [benchmark
package](https://github.com/xainag/xain/tree/master/benchmarks/benchmark) and the
package](https://github.com/xainag/xain-fl/tree/master/benchmarks/benchmark) and the
[benchmark
documentation](https://github.com/xainag/xain/blob/master/docs/quick.md#training).
documentation](https://github.com/xainag/xain-fl/blob/master/docs/quick.md#training).

## Install from source

For development we require some extra system dependencies:

- [clang-format 8+](https://clang.llvm.org/docs/ClangFormat.html)
- Linux: `sudo apt install clang-format`
- macOS: `brew install clang-format`
Expand All @@ -46,8 +47,8 @@ For development we require some extra system dependencies:
To clone this repository and to install the XAIN project, please execute the following commands:

```shell
$ git clone https://github.com/xainag/xain.git
$ cd xain
$ git clone https://github.com/xainag/xain-fl.git
$ cd xain-fl

$ python -m pip install -e .[dev]
```
Expand All @@ -64,6 +65,7 @@ $ pytest

The project documentation resides under `docs/`. To build the documentation
run:

```shell
$ cd docs/
$ make docs
Expand Down
57 changes: 28 additions & 29 deletions RELEASE_PROCESS.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,26 +17,26 @@ A release on git is just a tagged commit on the `master` branch.
Here we detail the process of creating a new Github release.

1. Create and merge a pull request that:
- increases the version number in
[`xain/__version__.py`](https://github.com/xainag/xain/blob/master/xain/__version__.py)
according the versioning schema.
- updates the
[`CHANGELOG.md`](https://github.com/xainag/xain/blob/master/CHANGELOG.md)
with all notable changes for the release.
- possibly update the `Development Status` classifiers in the
[`setup.py`](https://github.com/xainag/xain/blob/master/setup.py). You
can check supported classifiers in the [pypi
website](https://pypi.org/classifiers/).
2. Got to the [Github Releases tab](https://github.com/xainag/xain/releases)
- increases the version number in
[`xain_fl/__version__.py`](https://github.com/xainag/xain-fl/blob/master/xain_fl/__version__.py)
according the versioning schema.
- updates the
[`CHANGELOG.md`](https://github.com/xainag/xain-fl/blob/master/CHANGELOG.md)
with all notable changes for the release.
- possibly update the `Development Status` classifiers in the
[`setup.py`](https://github.com/xainag/xain-fl/blob/master/setup.py). You
can check supported classifiers in the [pypi
website](https://pypi.org/classifiers/).
2. Got to the [Github Releases tab](https://github.com/xainag/xain-fl/releases)
and create a new release:
- for the tag version use the version defined in 1. preceded by a `v`, e.g.
v0.3.2, and target master.
- for the release title use the same as the tag version.
- for the release description, copy the section from the
[`CHANGELOG.md`](https://github.com/xainag/xain/blob/master/CHANGELOG.md)
related to this version.
- possibly check the `This is a pre-release` check box.
- Publish the release.
- for the tag version use the version defined in 1. preceded by a `v`, e.g.
v0.3.2, and target master.
- for the release title use the same as the tag version.
- for the release description, copy the section from the
[`CHANGELOG.md`](https://github.com/xainag/xain-fl/blob/master/CHANGELOG.md)
related to this version.
- possibly check the `This is a pre-release` check box.
- Publish the release.

### How to publish a new release to PyPi

Expand All @@ -45,15 +45,14 @@ You can check more information in the [Python Packaging User
Guide](https://packaging.python.org/tutorials/packaging-projects/).

1. Checkout the current git tag e.g.
```bash
$ git checkout v0.3.2
```
```bash
$ git checkout v0.3.2
```
2. Generate the distribution archives:
```bash
$ python setup.py sdist bdist_wheel
```
```bash
$ python setup.py sdist bdist_wheel
```
3. Upload the distribution archives using the correct PyPi credentials:
```bash
$ python -m twine upload dist/*
```

```bash
$ python -m twine upload dist/*
```
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/aggregation.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
task_accuracies,
)
from benchmarks.helpers import storage
from xain.logger import get_logger
from xain_fl.logger import get_logger

FLAGS = flags.FLAGS

Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/final_task_accuracies.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from absl import flags, logging

from benchmarks.helpers import storage
from xain.types import PlotValues, XticksLabels, XticksLocations
from xain_fl.types import PlotValues, XticksLabels, XticksLocations

from .plot import plot
from .results import GroupResult, TaskResult
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
import pytest
from absl import flags

from xain.helpers import sha1
from xain_fl.helpers import sha1

from . import final_task_accuracies
from .results import TaskResult
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/learning_rate.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from absl import flags, logging

from benchmarks.helpers import storage
from xain.types import PlotValues
from xain_fl.types import PlotValues

from .plot import plot
from .results import GroupResult, TaskResult
Expand Down
4 changes: 2 additions & 2 deletions benchmarks/benchmark/aggregation/learning_rate_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
import pytest
from absl import flags

from xain.helpers import sha1
from xain.logger import get_logger
from xain_fl.helpers import sha1
from xain_fl.logger import get_logger

from . import learning_rate

Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/participant_hist.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from benchmarks.benchmark.aggregation.plot import plot_history_data
from benchmarks.benchmark.aggregation.results import GroupResult, TaskResult
from benchmarks.helpers.storage import create_output_subdir, fname_with_default_dir
from xain.types import Metrics
from xain_fl.types import Metrics

FLAGS = flags.FLAGS

Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/plot.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from numpy import ndarray

from benchmarks.helpers import storage
from xain.types import PlotValues
from xain_fl.types import PlotValues

matplotlib.use("AGG")

Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/results.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
from typing import List, Optional, cast

from benchmarks.helpers import storage
from xain.types import Metrics
from xain_fl.types import Metrics


class TaskResult(ABC):
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/task_accuracies.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from absl import flags, logging

from benchmarks.helpers import storage
from xain.types import PlotValues
from xain_fl.types import PlotValues

from .plot import plot
from .results import GroupResult, TaskResult
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/aggregation/task_accuracies_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

import pytest

from xain.helpers import sha1
from xain_fl.helpers import sha1

from . import task_accuracies

Expand Down
6 changes: 3 additions & 3 deletions benchmarks/benchmark/bench_ea.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@
from benchmarks.benchmark.exec import run
from benchmarks.benchmark.net import orig_cnn_compiled
from benchmarks.helpers import storage
from xain.datasets import load_splits
from xain.fl.coordinator.aggregate import EvoAgg
from xain.fl.coordinator.evaluator import Evaluator
from xain_fl.datasets import load_splits
from xain_fl.fl.coordinator.aggregate import EvoAgg
from xain_fl.fl.coordinator.evaluator import Evaluator

DEFAULT_R = 50
DEFAULT_E = 1 # Number of training epochs in each round
Expand Down
6 changes: 3 additions & 3 deletions benchmarks/benchmark/benchmark_test.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from benchmarks.ops.run import cores
from xain.datasets.dataset import config
from xain_fl.datasets.dataset import config

from .aggregation.aggregation import aggregations
from .benchmark import benchmarks
Expand Down Expand Up @@ -34,7 +34,7 @@ def test_valid_model_names():

def test_valid_dataset_names():
"""
Verify that all dataset names used in `Task` objects are available in `xain.datasets`
Verify that all dataset names used in `Task` objects are available in `xain_fl.datasets`
"""

# Prepare
Expand All @@ -50,7 +50,7 @@ def test_valid_dataset_names():

def test_valid_instance_cores():
"""
Verify that all dataset names used in `Task` objects are available in `xain.datasets`
Verify that all dataset names used in `Task` objects are available in `xain_fl.datasets`
"""

# Prepare
Expand Down
2 changes: 1 addition & 1 deletion benchmarks/benchmark/exec/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

from benchmarks.helpers import storage
from benchmarks.ops import results
from xain.datasets import load_splits
from xain_fl.datasets import load_splits

from . import run

Expand Down
Loading

0 comments on commit 481a9af

Please sign in to comment.