Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[exporter/elasticsearch] Add mapping mode bodymap #35637

Merged
merged 19 commits into from
Oct 17, 2024
Merged

[exporter/elasticsearch] Add mapping mode bodymap #35637

merged 19 commits into from
Oct 17, 2024

Conversation

mauri870
Copy link
Contributor

@mauri870 mauri870 commented Oct 7, 2024

Description

This PR implements a new mapping mode bodymap that works by serializing each LogRecord body as-is into a separate document for ingestion.

Fixes #35444

Testing

Documentation

@mauri870 mauri870 requested a review from a team as a code owner October 7, 2024 11:06
@mauri870 mauri870 requested a review from MovieStoreGuy October 7, 2024 11:06
@mauri870 mauri870 force-pushed the mapping-mode-mapbody branch from aedaddd to cb15832 Compare October 7, 2024 11:08
carsonip
carsonip reviewed Oct 8, 2024
View reviewed changes
Copy link
Contributor

@carsonip carsonip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first pass: looks good at a high level. Apparently it will still go through data stream routing without changing the body, which will risk actual index name and data_stream.* fields mismatch.

.chloggen/feature_elasticsearch_mapping_bodymap.yaml Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/README.md Outdated Show resolved Hide resolved
exporter/elasticsearchexporter/exporter_test.go Outdated Show resolved Hide resolved
@mauri870 mauri870 marked this pull request as draft October 10, 2024 14:39
@mauri870 mauri870 marked this pull request as ready for review October 10, 2024 18:58
@mauri870 mauri870 marked this pull request as draft October 14, 2024 17:35
carsonip
carsonip reviewed Oct 15, 2024
View reviewed changes
Copy link
Contributor

@carsonip carsonip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, lgtm except the handling when body is not a map.

exporter/elasticsearchexporter/README.md Outdated Show resolved Hide resolved
@mauri870 mauri870 marked this pull request as ready for review October 15, 2024 12:36
@mauri870 mauri870 requested a review from carsonip October 15, 2024 14:01
carsonip
carsonip approved these changes Oct 15, 2024
View reviewed changes
Copy link
Contributor

@carsonip carsonip left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

@mauri870
Copy link
Contributor Author

@andrzej-stencel Would be good to get your review on this. Thanks!

@andrzej-stencel andrzej-stencel merged commit e7ebc6e into open-telemetry:main Oct 17, 2024
158 checks passed
@github-actions github-actions bot added this to the next release milestone Oct 17, 2024
sbylica-splunk pushed a commit to sbylica-splunk/opentelemetry-collector-contrib that referenced this pull request Dec 17, 2024
@mauri870 @carsonip @sbylica-splunk
[exporter/elasticsearch] Add mapping mode bodymap (open-telemetry#35637)
996f206 
#### Description

This PR implements a new mapping mode `bodymap` that works by
serializing each LogRecord body as-is into a separate document for
ingestion.

Fixes open-telemetry#35444

#### Testing

#### Documentation

---------

Co-authored-by: Carson Ip <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@andrzej-stencel andrzej-stencel andrzej-stencel approved these changes

@carsonip carsonip carsonip approved these changes

@MovieStoreGuy MovieStoreGuy Awaiting requested review from MovieStoreGuy MovieStoreGuy is a code owner automatically assigned from open-telemetry/collector-contrib-approvers

@JaredTan95 JaredTan95 Awaiting requested review from JaredTan95

@lahsivjar lahsivjar Awaiting requested review from lahsivjar

@felixbarny felixbarny Awaiting requested review from felixbarny

Assignees

@jpkrohling jpkrohling

@fatsheep9146 fatsheep9146

Labels
exporter/elasticsearch
Projects
None yet
Milestone
v0.112.0
Development

Successfully merging this pull request may close these issues.

[exporter/elasticsearch] Ability to control final document structure for logs
6 participants
@mauri870 @felixbarny @carsonip @andrzej-stencel @jpkrohling @fatsheep9146