This repository has been archived by the owner on Nov 17, 2023. It is now read-only.
Static build instruction for MXNet in general #13914
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Thanks @lanking520 to enable MKLDNN backend for Maven build. Feel free to ping me if anything we can help. |
|
@mxnet-label-bot add [pr-awaiting-review] |
zachgk
reviewed
Jan 17, 2019
marcoabreu
reviewed
Jan 17, 2019
lanking520
force-pushed
the
static-build-instruction
branch
from
59a5631 to
8bb808b
Compare
zachgk
approved these changes
Jan 18, 2019
marcoabreu
approved these changes
Jan 18, 2019
| @@ -0,0 +1,33 @@ | |||
| # MXNet Publish Settings | |||
|
|
|||
| This folder contains the configuration of restricted node on Jenkins for the publish. It also contains a folder called `scala` that contains everything required for scala publish. In this `README`, we would bring a brief walkthrough of the Jenkins configuration as well as the usages of the scala deployment files. | |||
There was a problem hiding this comment.
Suggested change
| This folder contains the configuration of restricted node on Jenkins for the publish. It also contains a folder called `scala` that contains everything required for scala publish. In this `README`, we would bring a brief walkthrough of the Jenkins configuration as well as the usages of the scala deployment files. | |
| This folder contains the configuration for restricted nodes on Jenkins for the publishing MXNet artifacts. It also contains a folder called `scala` that contains everything required for publishing to Maven. In this `README`, we provide a brief walkthrough of the Jenkins configuration as well as the usage of the Scala deployment files. Python publishing is TBD. |
| This folder contains the configuration of restricted node on Jenkins for the publish. It also contains a folder called `scala` that contains everything required for scala publish. In this `README`, we would bring a brief walkthrough of the Jenkins configuration as well as the usages of the scala deployment files. | ||
|
|
||
| ## Jenkins | ||
| Currently, Jenkins contains three build stages, namely `Build Packages`, `Test Packages` and `Deploy Packages`. During the `build package` stages, all dependencies will be built and a Scala package would be created. In the second stage, the package created from the previous stage would move to this stage to specifically run the tests. In the final stage, the packages passed the test would be deployed by the instances. |
There was a problem hiding this comment.
Suggested change
| Currently, Jenkins contains three build stages, namely `Build Packages`, `Test Packages` and `Deploy Packages`. During the `build package` stages, all dependencies will be built and a Scala package would be created. In the second stage, the package created from the previous stage would move to this stage to specifically run the tests. In the final stage, the packages passed the test would be deployed by the instances. | |
| Currently, Jenkins contains three build stages, namely `Build Packages`, `Test Packages` and `Deploy Packages`. During the `build package` stages, all dependencies are built and a Scala package are created. In the second stage, the package created from the previous stage moves to this stage to specifically run the tests. In the final stage, the packages that pass the tests are deployed by the instances. |
|
|
||
| Apart from that, the script used to create the environment and publish are available under `ci/docker/install`: | ||
|
|
||
| - `ubuntu_publish.sh` install all required dependencies for Ubuntu 14.04 for publishing |
There was a problem hiding this comment.
Suggested change
| - `ubuntu_publish.sh` install all required dependencies for Ubuntu 14.04 for publishing | |
| - `ubuntu_publish.sh` installs all required dependencies for Ubuntu 14.04 for publishing |
| Apart from that, the script used to create the environment and publish are available under `ci/docker/install`: | ||
|
|
||
| - `ubuntu_publish.sh` install all required dependencies for Ubuntu 14.04 for publishing | ||
| - `ubuntu_base.sh` install minimum dependencies required to run the published packages |
There was a problem hiding this comment.
Suggested change
| - `ubuntu_base.sh` install minimum dependencies required to run the published packages | |
| - `ubuntu_base.sh` installs minimum dependencies required to run the published packages |
| - `ubuntu_base.sh` install minimum dependencies required to run the published packages | ||
|
|
||
| ## Scala publish | ||
| Currently Scala publish on Linux is fully supported on jenkins. The `scala/` folder contains all files needed to do the publish. Here is a breif instroduction of the files: |
There was a problem hiding this comment.
Suggested change
| Currently Scala publish on Linux is fully supported on jenkins. The `scala/` folder contains all files needed to do the publish. Here is a breif instroduction of the files: | |
| Currently Scala publish on Linux is fully supported on Jenkins. The `scala/` folder contains all files needed for publishing. Here is a brief introduction of the files: |
| @@ -0,0 +1,32 @@ | |||
| # MXNet Static build | |||
|
|
|||
| This folder contains the core script used to build the static library. This README would bring you the information and usages of the script in here. Please be aware, all of the scripts are designed to be run under the root folder. | |||
There was a problem hiding this comment.
Suggested change
| This folder contains the core script used to build the static library. This README would bring you the information and usages of the script in here. Please be aware, all of the scripts are designed to be run under the root folder. | |
| This folder contains the core script used to build the static library. This README provides information on how to use the scripts in this folder. Please be aware, all of the scripts are designed to be run under the root folder. |
| This folder contains the core script used to build the static library. This README would bring you the information and usages of the script in here. Please be aware, all of the scripts are designed to be run under the root folder. | ||
|
|
||
| ## `build.sh` | ||
| This script is a wrapper around `build_lib.sh` aimed to simplify the usage of it. It would automatically identify the system version, number of cores and all environment variable settings. Here are the examples you can run this script: |
There was a problem hiding this comment.
Suggested change
| This script is a wrapper around `build_lib.sh` aimed to simplify the usage of it. It would automatically identify the system version, number of cores and all environment variable settings. Here are the examples you can run this script: | |
| This script is a wrapper around `build_lib.sh. It simplifies the things by automatically identifing the system version, number of cores, and all environment variable settings. Here are examples you can run with this script: |
| As the result, users would have a complete static dependencies in `/staticdeps` in the root folder as well as a static-linked `libmxnet.so` file lives in `lib`. You can build your language binding by using the `libmxnet.so`. | ||
|
|
||
| ## `build_lib.sh` | ||
| This script would clone the most up-to-date master and build the MXNet backend with static library. In order to run that, you should have prepare the the following environment variable: |
There was a problem hiding this comment.
Suggested change
| This script would clone the most up-to-date master and build the MXNet backend with static library. In order to run that, you should have prepare the the following environment variable: | |
| This script clones the most up-to-date master and builds the MXNet backend with a static library. In order to run the static library, you must set the the following environment variables: |
| After running this script, you would have everything you need ready in the `/lib` folder. | ||
|
|
||
| ## `build_wheel.sh` | ||
| This script is used to build the python package as well as running a sanity test |
There was a problem hiding this comment.
Suggested change
| This script is used to build the python package as well as running a sanity test | |
| This script builds the python package. It also runs a sanity test. |
| - `ubuntu_publish.sh` install all required dependencies for Ubuntu 14.04 for publishing | ||
| - `ubuntu_base.sh` install minimum dependencies required to run the published packages | ||
|
|
||
| ## Scala publish |
There was a problem hiding this comment.
Suggested change
| ## Scala publish | |
| ## Scala Publishing |
stephenrawls
pushed a commit
to stephenrawls/incubator-mxnet
that referenced
this pull request
Feb 16, 2019
* update scripts and tutorial * add the static test for scala package * kill publish test * fix build issue * address comments
lanking520
added a commit
to lanking520/incubator-mxnet
that referenced
this pull request
Apr 26, 2019
* update scripts and tutorial * add the static test for scala package * kill publish test * fix build issue * address comments
lanking520
added a commit
to lanking520/incubator-mxnet
that referenced
this pull request
Apr 26, 2019
* update scripts and tutorial * add the static test for scala package * kill publish test * fix build issue * address comments
lanking520
added a commit
to lanking520/incubator-mxnet
that referenced
this pull request
Apr 30, 2019
* update scripts and tutorial * add the static test for scala package * kill publish test * fix build issue * address comments
zachgk
pushed a commit
to zachgk/incubator-mxnet
that referenced
this pull request
May 16, 2019
* update scripts and tutorial * add the static test for scala package * kill publish test * fix build issue * address comments
haohuanw
pushed a commit
to haohuanw/incubator-mxnet
that referenced
this pull request
Jun 23, 2019
* update scripts and tutorial * add the static test for scala package * kill publish test * fix build issue * address comments
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Adding instructions for Static build and a static build test stage in CI. Also make the mkldnn as a default on Maven publish and enable signal_handler for segfault.
@frankfliu @szha @marcoabreu @zachgk @gigasquid
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.