Add interfaces for workspace, other yamls. #212

jcscottiii · 2016-08-01T13:53:43Z

To enable inter-operability between different versions of different things like standards and certifications (opencontrol.yaml and component.yaml already has it) and easy mocking, this patch introduces the use of more interfaces. Also, this patch introduces workspace interface which enables for external plugins to easily test by just using the interface to mock instead of worrying about fixtures.

Major Additions:

Added example plugin, plugin tests with mocks and README.
Added interfaces for Certifications, Control, Standard, Workspace to the common package
Moved interfaces Component, Satisfies, and Section to the common package
Added importable mocks for Component, Satisfies, Certifications, Control, Section, Standard, Workspace
Moved Standard to it's own package for future versioning and loading by versions.
- A lot of plumbing is still needed here like adding a special UnmarshalYAML but the whole point of moving it was to move all yaml files into their own package, out of the base lib package.
Moved Certification to it's own package for future versioning and loading by versions.
- A lot of plumbing is still needed here like adding a special UnmarshalYAML but the whole point of moving it was to move all yaml files into their own package, out of the base lib package

Other changes:

Removed the use of callbacks
- This will help with mocking
Made the maps Standards and Components and their methods private.
- This makes it more clear for people who import the lib package, which things they can use. If something is private, they won't see it.
Fixed a lot of golint problems
- When I moved files around, the old golint problems became new again so I had to fix them.

Actual logic changes:

For versions 2.0.0, and 3.0.0 of component, GetImplementationStatuses returned an empty array. It should return the implementation status if it's there.
- Can probably be moved out of this PR but it's a small change compared to the rest.

For future PR:

Add verifications to common package.
- This will allow for workspace to also be added to common without creating a circular dependency chain.

Excerpt from `lib/README.md`:

Lib Package

This package (and the sub packages) are the only packages that should be imported by external projects. This lib package is very helpful for those writing plugins to extend the functionality of Masonry.

`Workspace`

Workspace is the starting point for all the information available to plugins. For those using Go plugins, using the interface methods are all you need. It represents all the information collected into the opencontrols folder after running the get command.

YAML files

The yaml files are represented by:

Components
Certifications
Standards

Result Data

Justifications is a data structure that is not represented in yaml but rather a post-processed map of data to help quickly getting component data for a particular control name - standard name combination.

Plugin Developer Guide

Developers should not have to worry about loading real data / workspaces for their unit tests (they should for integration tests).

There is an example of developing your Go plugin and tests in exampleplugin/example.go and exampleplugin/example_test.go respectively.

codecov-io · 2016-08-02T14:04:49Z

Current coverage is 88.40% (diff: 82.83%)

Merging #212 into master will decrease coverage by 4.16%

@@             master       #212   diff @@
==========================================
  Files            26         31     +5   
  Lines           888        923    +35   
  Methods           0          0          
  Messages          0          0          
  Branches          0          0          
==========================================
- Hits            822        816     -6   
- Misses           52         91    +39   
- Partials         14         16     +2

Powered by Codecov. Last update a0746d3...11c8c70

Having the interfaces for each type in it's own package was fine when it is the only one imported via it's "base" package. However, when you import all of them as in workspace, you have to start aliasing the package names and that's just sloppy.

jcscottiii · 2016-08-03T15:42:35Z

@afeld yeppp.. I did it again. this is a mega elephant patch. A lot of it is moving files around though. Details above. This PR was one of those that started off one way then turned 90 degrees to something else so it was hard to break it up. Now that it's done, I can do that now. Let me know and I'll start to break this up later. Not a huge priority now so I'll get to it later probably.

afeld · 2016-08-10T16:10:25Z

exampleplugin/example_test.go

+import (
+	"testing"
+	"github.com/opencontrol/compliance-masonry/lib/mocks"
+	commonMocks"github.com/opencontrol/compliance-masonry/lib/common/mocks"


Needs a space (right?)

whoops! will fix

afeld · 2016-08-11T03:02:58Z

A few pieces of general feedback:

Wow, yeah, this was massive...let's not do that again 😬
Love the example plugin and its documentation! How about we split its addition out to its own pull request?
Not sure that I understand the common package...maybe it could have a more descriptive name?

Sorry for the slow turnaround on the review, by the way!

For versions 2.0.0, and 3.0.0 of component, GetImplementationStatuses returned an empty array. It should return the implementation status if it's there. Can probably be moved out of this PR

If it's not hard, that would be good!

jcscottiii · 2016-08-11T14:00:54Z

Not sure that I understand the common package...maybe it could have a more descriptive name?

Yeah, I agree. There are some things like Component and Standard there so it can read as common.Component and the Workspace is supposed to be able to return these common Components and common Standards (regardless of which version it is; 3.0.0 vs. 3.1.0) but there were some existing things like Entry that don't belong in the same package. Will look into moving out some of that out and add documentation for that.

afeld · 2016-08-19T18:42:13Z

@jcscottiii Are you planning to rebase this one (sooner or later) or should we just close it?

afeld · 2016-10-03T21:50:01Z

Bump!

jcscottiii · 2016-10-03T21:58:38Z

Let's close this. The other recent PRs have been just parts of this one.
The only PRs yet to be made are 1) generate the mocks and 2) example plugin.

On Mon, Oct 3, 2016, 5:50 PM Aidan Feldman [email protected] wrote:

Bump!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
#212 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AHbZgrvrYvbFio05ytQd3m24ibrjwk5Kks5qwXiLgaJpZM4JZmxL
.

jcscottiii · 2016-10-04T14:36:46Z

Split out to #213 #214 #215 #216 #217 #226 #227 #228

James C. Scott added 12 commits July 29, 2016 10:23

Refactor pt.1 Remove callbacks and add Workspace interface

a506f4b

Add Standards interface to workspace

ee515d4

Add interface for certification

7790d94

Move errors to common

dd6f629

Move control to common

c2a247a

Add version 1.0.0 of standard

5815593

Fix renders with new interface methods

e0cc51d

Make standards container methods private

6dad26a

Move around certifications tests

81076bb

Fix gitbook tests

e0f6a17

Optimize counting for test

953173b

Fix standards tests

ac1ff4d

James C. Scott added 15 commits August 2, 2016 10:50

Make componentsMap private like standardsMap

d31854c

Move standards interface to common package

fd2e9d2

Move certification interface to common

f7e7f29

Move LoadComponents to components package

2ea8a8d

Make LocalWorkspace private

33328c2

Fix golint errors from codeclimate

11c8c70

Add some tests to the localworkspace methods

5884e27

add test for standard v1

e10e6d2

Add tests for certification v1

93759b4

Add mocks

42d8925

add tests for componentsMap

81180aa

Fix import for assert

7ae9e50

Add example plugin and plugin README

2c9f6f1

Fix implementation statuses for 2_0_0 and 3_0_0

fefa6a9

jcscottiii changed the title ~~Add interfaces for workspace, other yamls. [WIP]~~ Add interfaces for workspace, other yamls. Aug 3, 2016

afeld reviewed Aug 10, 2016
View reviewed changes

This was referenced Aug 19, 2016

Move Components interface to common package #213

Merged

Move standards interface #214

Merged

afeld closed this Oct 4, 2016

afeld deleted the workspace-pt2 branch October 4, 2016 03:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add interfaces for workspace, other yamls. #212

Add interfaces for workspace, other yamls. #212

jcscottiii commented Aug 1, 2016 •

edited

Loading

codecov-io commented Aug 2, 2016 •

edited

Loading

jcscottiii commented Aug 3, 2016

afeld Aug 10, 2016

jcscottiii Aug 11, 2016

afeld commented Aug 11, 2016

jcscottiii commented Aug 11, 2016 •

edited

Loading

afeld commented Aug 19, 2016

afeld commented Oct 3, 2016

jcscottiii commented Oct 3, 2016

jcscottiii commented Oct 4, 2016

Add interfaces for workspace, other yamls. #212

Add interfaces for workspace, other yamls. #212

Conversation

jcscottiii commented Aug 1, 2016 • edited Loading

Major Additions:

Other changes:

Actual logic changes:

Excerpt from lib/README.md:

Lib Package

Workspace

YAML files

Result Data

Plugin Developer Guide

codecov-io commented Aug 2, 2016 • edited Loading

Current coverage is 88.40% (diff: 82.83%)

jcscottiii commented Aug 3, 2016

afeld Aug 10, 2016

Choose a reason for hiding this comment

jcscottiii Aug 11, 2016

Choose a reason for hiding this comment

afeld commented Aug 11, 2016

jcscottiii commented Aug 11, 2016 • edited Loading

afeld commented Aug 19, 2016

afeld commented Oct 3, 2016

jcscottiii commented Oct 3, 2016

jcscottiii commented Oct 4, 2016

jcscottiii commented Aug 1, 2016 •

edited

Loading

Excerpt from `lib/README.md`:

`Workspace`

codecov-io commented Aug 2, 2016 •

edited

Loading

jcscottiii commented Aug 11, 2016 •

edited

Loading