Fix broken marker file update

[dosumis]

The most recent RF marker file commit has issues with duplicate nodes, with different markers + differences in syntax (curie vs unprefixed)

Taxonomy_node_ID	clusterName	Markers Delimited
CS202002013_123	GABAergic	ensembl:ENSMUSG00000070880|ensembl:ENSMUSG00000098326
CS202002013_123		ENSMUSG00000037610|ENSMUSG00000053519

@BAevermann Can you look into this & fix?

[hkir-dev]

Duplicate Id check added to validator. Latest validator code merged to master so now on validation report mails should be more accurate.

[hkir-dev]

Rolled back the marker data in the main branch to the previous stable version. Moved marker data to be fixed to the marker_update branch. We can fix and make pull requests from this branch: https://github.com/obophenotype/brain_data_standards_ontologies/blob/marker_update/src/markers/CS202002013_markers.tsv

[hkir-dev]

Errors in the current marker file are:

Validation Report

=== Table Structure Checks :
Invalid column names: ['Taxonomy_node_ID', 'clusterName', 'Markers Delimited'] in file CS202002013_markers.tsv. Expected columns are: ['Taxonomy_node_ID', 'clusterName', 'Markers']

=== Marker Nodes' Dendrogram Existence Checks :
Invalid Taxonomy_node_ID 'CS202002013_235' in the marker file (CS202002013_markers.tsv). Id not exist in the dendrogram.
Invalid Taxonomy_node_ID 'CS202002013_233' in the marker file (CS202002013_markers.tsv). Id not exist in the dendrogram.
Invalid Taxonomy_node_ID 'CS202002013_232' in the marker file (CS202002013_markers.tsv). Id not exist in the dendrogram.
Invalid Taxonomy_node_ID 'CS202002013_250' in the marker file (CS202002013_markers.tsv). Id not exist in the dendrogram.
Redundant Taxonomy_node_ID 'CS202002013_123' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_179' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_138' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_203' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_197' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_125' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_133' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_193' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_222' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_229' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_212' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_103' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_183' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_189' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_207' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_210' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_91' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_192' in the marker file (CS202002013_markers.tsv).
Redundant Taxonomy_node_ID 'CS202002013_209' in the marker file (CS202002013_markers.tsv).

=== Marker Name Checks :
Invalid marker 'ENSMUSG00000037610' in file 'CS202002013_markers.tsv' with key 'CS202002013_123'.
Invalid marker 'ENSMUSG00000053519' in file 'CS202002013_markers.tsv' with key 'CS202002013_123'.
Invalid marker 'ENSMUSG00000053025' in file 'CS202002013_markers.tsv' with key 'CS202002013_179'.
Invalid marker 'ENSMUSG00000070570' in file 'CS202002013_markers.tsv' with key 'CS202002013_179'.
Invalid marker 'ENSMUSG00000046178' in file 'CS202002013_markers.tsv' with key 'CS202002013_179'.
Invalid marker 'ENSMUSG00000021318' in file 'CS202002013_markers.tsv' with key 'CS202002013_210'.
Invalid marker 'ENSMUSG00000036949' in file 'CS202002013_markers.tsv' with key 'CS202002013_210'.
Invalid marker 'ENSMUSG00000030237' in file 'CS202002013_markers.tsv' with key 'CS202002013_103'.
Invalid marker 'ENSMUSG00000039167' in file 'CS202002013_markers.tsv' with key 'CS202002013_103'.
Invalid marker 'ENSMUSG00000059187' in file 'CS202002013_markers.tsv' with key 'CS202002013_183'.
Invalid marker 'ENSMUSG00000028222' in file 'CS202002013_markers.tsv' with key 'CS202002013_183'.
Invalid marker 'ENSMUSG00000056427' in file 'CS202002013_markers.tsv' with key 'CS202002013_183'.
Invalid marker 'ENSMUSG00000022206' in file 'CS202002013_markers.tsv' with key 'CS202002013_193'.
Invalid marker 'ENSMUSG00000098760' in file 'CS202002013_markers.tsv' with key 'CS202002013_193'.
Invalid marker 'ENSMUSG00000116610' in file 'CS202002013_markers.tsv' with key 'CS202002013_189'.
Invalid marker 'ENSMUSG00000059203' in file 'CS202002013_markers.tsv' with key 'CS202002013_189'.
Invalid marker 'ENSMUSG00000021998' in file 'CS202002013_markers.tsv' with key 'CS202002013_207'.
Invalid marker 'ENSMUSG00000096972' in file 'CS202002013_markers.tsv' with key 'CS202002013_207'.
Invalid marker 'ENSMUSG00000027849' in file 'CS202002013_markers.tsv' with key 'CS202002013_197'.
Invalid marker 'ENSMUSG00000078591' in file 'CS202002013_markers.tsv' with key 'CS202002013_197'.
Invalid marker 'ENSMUSG00000020651' in file 'CS202002013_markers.tsv' with key 'CS202002013_192'.
Invalid marker 'ENSMUSG00000016918' in file 'CS202002013_markers.tsv' with key 'CS202002013_192'.
Invalid marker 'ENSMUSG00000019997' in file 'CS202002013_markers.tsv' with key 'CS202002013_203'.
Invalid marker 'ENSMUSG00000039714' in file 'CS202002013_markers.tsv' with key 'CS202002013_203'.
Invalid marker 'ENSMUSG00000075270' in file 'CS202002013_markers.tsv' with key 'CS202002013_125'.
Invalid marker 'ENSMUSG00000029819' in file 'CS202002013_markers.tsv' with key 'CS202002013_125'.
Invalid marker 'ENSMUSG00000027210' in file 'CS202002013_markers.tsv' with key 'CS202002013_209'.
Invalid marker 'ENSMUSG00000026288' in file 'CS202002013_markers.tsv' with key 'CS202002013_229'.
Invalid marker 'ENSMUSG00000031425' in file 'CS202002013_markers.tsv' with key 'CS202002013_212'.
Invalid marker 'ENSMUSG00000033740' in file 'CS202002013_markers.tsv' with key 'CS202002013_212'.
Invalid marker 'ENSMUSG00000017978' in file 'CS202002013_markers.tsv' with key 'CS202002013_133'.
Invalid marker 'ENSMUSG00000017897' in file 'CS202002013_markers.tsv' with key 'CS202002013_222'.
Invalid marker 'ENSMUSG00000039004' in file 'CS202002013_markers.tsv' with key 'CS202002013_222'.
Invalid marker 'ENSMUSG00000019772' in file 'CS202002013_markers.tsv' with key 'CS202002013_138'.
Invalid marker 'ENSMUSG00000039954' in file 'CS202002013_markers.tsv' with key 'CS202002013_91'.
Invalid marker 'ENSMUSG00000029231' in file 'CS202002013_markers.tsv' with key 'CS202002013_91'.
Invalid marker 'ENSMUSG00000022860' in file 'CS202002013_markers.tsv' with key 'CS202002013_28'.
Invalid marker 'ENSMUSG00000047495' in file 'CS202002013_markers.tsv' with key 'CS202002013_121'.
Invalid marker 'ENSMUSG00000025576' in file 'CS202002013_markers.tsv' with key 'CS202002013_121'.
Invalid marker 'ENSMUSG00000025576' in file 'CS202002013_markers.tsv' with key 'CS202002013_122'.
Invalid marker 'ENSMUSG00000047495' in file 'CS202002013_markers.tsv' with key 'CS202002013_122'.
Invalid marker 'ENSMUSG00000094083' in file 'CS202002013_markers.tsv' with key 'CS202002013_150'.
Invalid marker 'ENSMUSG00000046178' in file 'CS202002013_markers.tsv' with key 'CS202002013_150'.
Invalid marker 'ENSMUSG00000010175' in file 'CS202002013_markers.tsv' with key 'CS202002013_124'.
Invalid marker 'ENSMUSG00000052551' in file 'CS202002013_markers.tsv' with key 'CS202002013_124'.
Invalid marker 'ENSMUSG00000033063' in file 'CS202002013_markers.tsv' with key 'CS202002013_151'.
Invalid marker 'ENSMUSG00000094083' in file 'CS202002013_markers.tsv' with key 'CS202002013_151'.
Invalid marker 'ENSMUSG00000100851' in file 'CS202002013_markers.tsv' with key 'CS202002013_180'.
Invalid marker 'ENSMUSG00000059173' in file 'CS202002013_markers.tsv' with key 'CS202002013_180'.
Invalid marker 'ENSMUSG00000019772' in file 'CS202002013_markers.tsv' with key 'CS202002013_132'.
Invalid marker 'ENSMUSG00000048988' in file 'CS202002013_markers.tsv' with key 'CS202002013_152'.
Invalid marker 'ENSMUSG00000004366' in file 'CS202002013_markers.tsv' with key 'CS202002013_152'.
Invalid marker 'ENSMUSG00000005716' in file 'CS202002013_markers.tsv' with key 'CS202002013_168'.
Invalid marker 'ENSMUSG00000087301' in file 'CS202002013_markers.tsv' with key 'CS202002013_168'.
Invalid marker 'ENSMUSG00000038331' in file 'CS202002013_markers.tsv' with key 'CS202002013_181'.
Invalid marker 'ENSMUSG00000036264' in file 'CS202002013_markers.tsv' with key 'CS202002013_181'.
Invalid marker 'ENSMUSG00000063626' in file 'CS202002013_markers.tsv' with key 'CS202002013_181'.
Invalid marker 'ENSMUSG00000027849' in file 'CS202002013_markers.tsv' with key 'CS202002013_196'.
Invalid marker 'ENSMUSG00000078591' in file 'CS202002013_markers.tsv' with key 'CS202002013_196'.
Invalid marker 'ENSMUSG00000048988' in file 'CS202002013_markers.tsv' with key 'CS202002013_153'.
Invalid marker 'ENSMUSG00000004366' in file 'CS202002013_markers.tsv' with key 'CS202002013_153'.
Invalid marker 'ENSMUSG00000091002' in file 'CS202002013_markers.tsv' with key 'CS202002013_169'.
Invalid marker 'ENSMUSG00000031558' in file 'CS202002013_markers.tsv' with key 'CS202002013_169'.
Invalid marker 'ENSMUSG00000010461' in file 'CS202002013_markers.tsv' with key 'CS202002013_171'.
Invalid marker 'ENSMUSG00000052353' in file 'CS202002013_markers.tsv' with key 'CS202002013_171'.
Invalid marker 'ENSMUSG00000036264' in file 'CS202002013_markers.tsv' with key 'CS202002013_182'.
Invalid marker 'ENSMUSG00000038331' in file 'CS202002013_markers.tsv' with key 'CS202002013_182'.
Invalid marker 'ENSMUSG00000036256' in file 'CS202002013_markers.tsv' with key 'CS202002013_220'.
Invalid marker 'ENSMUSG00000019772' in file 'CS202002013_markers.tsv' with key 'CS202002013_141'.
Invalid marker 'ENSMUSG00000021680' in file 'CS202002013_markers.tsv' with key 'CS202002013_158'.
Invalid marker 'ENSMUSG00000048988' in file 'CS202002013_markers.tsv' with key 'CS202002013_158'.
Invalid marker 'ENSMUSG00000010461' in file 'CS202002013_markers.tsv' with key 'CS202002013_172'.
Invalid marker 'ENSMUSG00000052353' in file 'CS202002013_markers.tsv' with key 'CS202002013_172'.
Invalid marker 'ENSMUSG00000059203' in file 'CS202002013_markers.tsv' with key 'CS202002013_185'.
Invalid marker 'ENSMUSG00000038331' in file 'CS202002013_markers.tsv' with key 'CS202002013_185'.
Invalid marker 'ENSMUSG00000029648' in file 'CS202002013_markers.tsv' with key 'CS202002013_219'.
Invalid marker 'ENSMUSG00000039167' in file 'CS202002013_markers.tsv' with key 'CS202002013_219'.
Invalid marker 'ENSMUSG00000019772' in file 'CS202002013_markers.tsv' with key 'CS202002013_142'.
Invalid marker 'ENSMUSG00000021680' in file 'CS202002013_markers.tsv' with key 'CS202002013_160'.
Invalid marker 'ENSMUSG00000048988' in file 'CS202002013_markers.tsv' with key 'CS202002013_160'.
Invalid marker 'ENSMUSG00000010461' in file 'CS202002013_markers.tsv' with key 'CS202002013_173'.
Invalid marker 'ENSMUSG00000052353' in file 'CS202002013_markers.tsv' with key 'CS202002013_173'.
Invalid marker 'ENSMUSG00000059203' in file 'CS202002013_markers.tsv' with key 'CS202002013_186'.
Invalid marker 'ENSMUSG00000038331' in file 'CS202002013_markers.tsv' with key 'CS202002013_186'.
Invalid marker 'ENSMUSG00000029563' in file 'CS202002013_markers.tsv' with key 'CS202002013_198'.
Invalid marker 'ENSMUSG00000078591' in file 'CS202002013_markers.tsv' with key 'CS202002013_198'.
Invalid marker 'ENSMUSG00000021508' in file 'CS202002013_markers.tsv' with key 'CS202002013_126'.
Invalid marker 'ENSMUSG00000042453' in file 'CS202002013_markers.tsv' with key 'CS202002013_126'.
Invalid marker 'ENSMUSG00000051111' in file 'CS202002013_markers.tsv' with key 'CS202002013_129'.
Invalid marker 'ENSMUSG00000075270' in file 'CS202002013_markers.tsv' with key 'CS202002013_129'.
Invalid marker 'ENSMUSG00000030170' in file 'CS202002013_markers.tsv' with key 'CS202002013_145'.
Invalid marker 'ENSMUSG00000019772' in file 'CS202002013_markers.tsv' with key 'CS202002013_145'.
Invalid marker 'ENSMUSG00000062151' in file 'CS202002013_markers.tsv' with key 'CS202002013_154'.
Invalid marker 'ENSMUSG00000059203' in file 'CS202002013_markers.tsv' with key 'CS202002013_154'.
Invalid marker 'ENSMUSG00000004366' in file 'CS202002013_markers.tsv' with key 'CS202002013_162'.
Invalid marker 'ENSMUSG00000048988' in file 'CS202002013_markers.tsv' with key 'CS202002013_162'.
Invalid marker 'ENSMUSG00000055761' in file 'CS202002013_markers.tsv' with key 'CS202002013_162'.
Invalid marker 'ENSMUSG00000010461' in file 'CS202002013_markers.tsv' with key 'CS202002013_174'.
Invalid marker 'ENSMUSG00000052353' in file 'CS202002013_markers.tsv' with key 'CS202002013_174'.
Invalid marker 'ENSMUSG00000036192' in file 'CS202002013_markers.tsv' with key 'CS202002013_187'.
Invalid marker 'ENSMUSG00000034687' in file 'CS202002013_markers.tsv' with key 'CS202002013_187'.
Invalid marker 'ENSMUSG00000036264' in file 'CS202002013_markers.tsv' with key 'CS202002013_187'.
Invalid marker 'ENSMUSG00000029563' in file 'CS202002013_markers.tsv' with key 'CS202002013_199'.
Invalid marker 'ENSMUSG00000078591' in file 'CS202002013_markers.tsv' with key 'CS202002013_199'.
Invalid marker 'ENSMUSG00000116029' in file 'CS202002013_markers.tsv' with key 'CS202002013_204'.
Invalid marker 'ENSMUSG00000019997' in file 'CS202002013_markers.tsv' with key 'CS202002013_204'.
Invalid marker 'ENSMUSG00000021508' in file 'CS202002013_markers.tsv' with key 'CS202002013_127'.
Invalid marker 'ENSMUSG00000042453' in file 'CS202002013_markers.tsv' with key 'CS202002013_127'.
Invalid marker 'ENSMUSG00000075270' in file 'CS202002013_markers.tsv' with key 'CS202002013_130'.
Invalid marker 'ENSMUSG00000005672' in file 'CS202002013_markers.tsv' with key 'CS202002013_130'.
Invalid marker 'ENSMUSG00000028004' in file 'CS202002013_markers.tsv' with key 'CS202002013_135'.
Invalid marker 'ENSMUSG00000029101' in file 'CS202002013_markers.tsv' with key 'CS202002013_135'.
Invalid marker 'ENSMUSG00000022371' in file 'CS202002013_markers.tsv' with key 'CS202002013_139'.
Invalid marker 'ENSMUSG00000038718' in file 'CS202002013_markers.tsv' with key 'CS202002013_139'.
Invalid marker 'ENSMUSG00000022425' in file 'CS202002013_markers.tsv' with key 'CS202002013_139'.
Invalid marker 'ENSMUSG00000019772' in file 'CS202002013_markers.tsv' with key 'CS202002013_143'.
Invalid marker 'ENSMUSG00000049796' in file 'CS202002013_markers.tsv' with key 'CS202002013_143'.
Invalid marker 'ENSMUSG00000000805' in file 'CS202002013_markers.tsv' with key 'CS202002013_143'.
Invalid marker 'ENSMUSG00000048776' in file 'CS202002013_markers.tsv' with key 'CS202002013_146'.
Invalid marker 'ENSMUSG00000030170' in file 'CS202002013_markers.tsv' with key 'CS202002013_146'.
Invalid marker 'ENSMUSG00000019772' in file 'CS202002013_markers.tsv' with key 'CS202002013_146'.
Invalid marker 'ENSMUSG00000027400' in file 'CS202002013_markers.tsv' with key 'CS202002013_156'.
Invalid marker 'ENSMUSG00000062151' in file 'CS202002013_markers.tsv' with key 'CS202002013_156'.
Invalid marker 'ENSMUSG00000024598' in file 'CS202002013_markers.tsv' with key 'CS202002013_159'.
Invalid marker 'ENSMUSG00000031997' in file 'CS202002013_markers.tsv' with key 'CS202002013_159'.
Invalid marker 'ENSMUSG00000038156' in file 'CS202002013_markers.tsv' with key 'CS202002013_159'.
Invalid marker 'ENSMUSG00000004366' in file 'CS202002013_markers.tsv' with key 'CS202002013_163'.
Invalid marker 'ENSMUSG00000003746' in file 'CS202002013_markers.tsv' with key 'CS202002013_163'.
Invalid marker 'ENSMUSG00000042453' in file 'CS202002013_markers.tsv' with key 'CS202002013_163'.
Invalid marker 'ENSMUSG00000049796' in file 'CS202002013_markers.tsv' with key 'CS202002013_165'.
Invalid marker 'ENSMUSG00000036019' in file 'CS202002013_markers.tsv' with key 'CS202002013_165'.
Invalid marker 'ENSMUSG00000031841' in file 'CS202002013_markers.tsv' with key 'CS202002013_165'.
Invalid marker 'ENSMUSG00000034324' in file 'CS202002013_markers.tsv' with key 'CS202002013_175'.
Invalid marker 'ENSMUSG00000050830' in file 'CS202002013_markers.tsv' with key 'CS202002013_175'.
Invalid marker 'ENSMUSG00000038048' in file 'CS202002013_markers.tsv' with key 'CS202002013_175'.
Invalid marker 'ENSMUSG00000090125' in file 'CS202002013_markers.tsv' with key 'CS202002013_194'.
Invalid marker 'ENSMUSG00000021541' in file 'CS202002013_markers.tsv' with key 'CS202002013_194'.
Invalid marker 'ENSMUSG00000068196' in file 'CS202002013_markers.tsv' with key 'CS202002013_194'.
Invalid marker 'ENSMUSG00000029563' in file 'CS202002013_markers.tsv' with key 'CS202002013_200'.
Invalid marker 'ENSMUSG00000078591' in file 'CS202002013_markers.tsv' with key 'CS202002013_200'.
Invalid marker 'ENSMUSG00000019997' in file 'CS202002013_markers.tsv' with key 'CS202002013_205'.
Invalid marker 'ENSMUSG00000039714' in file 'CS202002013_markers.tsv' with key 'CS202002013_205'.
Invalid marker 'ENSMUSG00000115529' in file 'CS202002013_markers.tsv' with key 'CS202002013_213'.
Invalid marker 'ENSMUSG00000020422' in file 'CS202002013_markers.tsv' with key 'CS202002013_213'.
Invalid marker 'ENSMUSG00000032841' in file 'CS202002013_markers.tsv' with key 'CS202002013_216'.
Invalid marker 'ENSMUSG00000032517' in file 'CS202002013_markers.tsv' with key 'CS202002013_216'.
Invalid marker 'ENSMUSG00000032796' in file 'CS202002013_markers.tsv' with key 'CS202002013_223'.
Invalid marker 'ENSMUSG00000030108' in file 'CS202002013_markers.tsv' with key 'CS202002013_223'.
Invalid marker 'ENSMUSG00000010122' in file 'CS202002013_markers.tsv' with key 'CS202002013_226'.
Invalid marker 'ENSMUSG00000028487' in file 'CS202002013_markers.tsv' with key 'CS202002013_226'.
Invalid marker 'ENSMUSG00000021665' in file 'CS202002013_markers.tsv' with key 'CS202002013_230'.
Invalid marker 'ENSMUSG00000029231' in file 'CS202002013_markers.tsv' with key 'CS202002013_120'.
Invalid marker 'ENSMUSG00000039954' in file 'CS202002013_markers.tsv' with key 'CS202002013_120'.
Invalid marker 'ENSMUSG00000049001' in file 'CS202002013_markers.tsv' with key 'CS202002013_128'.
Invalid marker 'ENSMUSG00000021508' in file 'CS202002013_markers.tsv' with key 'CS202002013_128'.
Invalid marker 'ENSMUSG00000049001' in file 'CS202002013_markers.tsv' with key 'CS202002013_131'.
Invalid marker 'ENSMUSG00000075270' in file 'CS202002013_markers.tsv' with key 'CS202002013_131'.
Invalid marker 'ENSMUSG00000048967' in file 'CS202002013_markers.tsv' with key 'CS202002013_134'.
Invalid marker 'ENSMUSG00000059049' in file 'CS202002013_markers.tsv' with key 'CS202002013_134'.
Invalid marker 'ENSMUSG00000028364' in file 'CS202002013_markers.tsv' with key 'CS202002013_136'.
Invalid marker 'ENSMUSG00000058897' in file 'CS202002013_markers.tsv' with key 'CS202002013_136'.
Invalid marker 'ENSMUSG00000028004' in file 'CS202002013_markers.tsv' with key 'CS202002013_137'.
Invalid marker 'ENSMUSG00000029101' in file 'CS202002013_markers.tsv' with key 'CS202002013_137'.
Invalid marker 'ENSMUSG00000069911' in file 'CS202002013_markers.tsv' with key 'CS202002013_140'.
Invalid marker 'ENSMUSG00000084908' in file 'CS202002013_markers.tsv' with key 'CS202002013_140'.
Invalid marker 'ENSMUSG00000037362' in file 'CS202002013_markers.tsv' with key 'CS202002013_140'.
Invalid marker 'ENSMUSG00000028121' in file 'CS202002013_markers.tsv' with key 'CS202002013_144'.
Invalid marker 'ENSMUSG00000019772' in file 'CS202002013_markers.tsv' with key 'CS202002013_144'.
Invalid marker 'ENSMUSG00000021662' in file 'CS202002013_markers.tsv' with key 'CS202002013_144'.
Invalid marker 'ENSMUSG00000021919' in file 'CS202002013_markers.tsv' with key 'CS202002013_147'.
Invalid marker 'ENSMUSG00000023945' in file 'CS202002013_markers.tsv' with key 'CS202002013_147'.
Invalid marker 'ENSMUSG00000019772' in file 'CS202002013_markers.tsv' with key 'CS202002013_148'.
Invalid marker 'ENSMUSG00000057378' in file 'CS202002013_markers.tsv' with key 'CS202002013_148'.
Invalid marker 'ENSMUSG00000024211' in file 'CS202002013_markers.tsv' with key 'CS202002013_148'.
Invalid marker 'ENSMUSG00000110468' in file 'CS202002013_markers.tsv' with key 'CS202002013_149'.
Invalid marker 'ENSMUSG00000042581' in file 'CS202002013_markers.tsv' with key 'CS202002013_149'.
Invalid marker 'ENSMUSG00000070366' in file 'CS202002013_markers.tsv' with key 'CS202002013_155'.
Invalid marker 'ENSMUSG00000059203' in file 'CS202002013_markers.tsv' with key 'CS202002013_155'.
Invalid marker 'ENSMUSG00000055214' in file 'CS202002013_markers.tsv' with key 'CS202002013_157'.
Invalid marker 'ENSMUSG00000039579' in file 'CS202002013_markers.tsv' with key 'CS202002013_157'.
Invalid marker 'ENSMUSG00000068220' in file 'CS202002013_markers.tsv' with key 'CS202002013_161'.
Invalid marker 'ENSMUSG00000031997' in file 'CS202002013_markers.tsv' with key 'CS202002013_161'.
Invalid marker 'ENSMUSG00000024990' in file 'CS202002013_markers.tsv' with key 'CS202002013_161'.
Invalid marker 'ENSMUSG00000004366' in file 'CS202002013_markers.tsv' with key 'CS202002013_164'.
Invalid marker 'ENSMUSG00000027965' in file 'CS202002013_markers.tsv' with key 'CS202002013_164'.
Invalid marker 'ENSMUSG00000114028' in file 'CS202002013_markers.tsv' with key 'CS202002013_166'.
Invalid marker 'ENSMUSG00000049796' in file 'CS202002013_markers.tsv' with key 'CS202002013_166'.
Invalid marker 'ENSMUSG00000025400' in file 'CS202002013_markers.tsv' with key 'CS202002013_167'.
Invalid marker 'ENSMUSG00000018417' in file 'CS202002013_markers.tsv' with key 'CS202002013_167'.
Invalid marker 'ENSMUSG00000036019' in file 'CS202002013_markers.tsv' with key 'CS202002013_167'.
Invalid marker 'ENSMUSG00000000214' in file 'CS202002013_markers.tsv' with key 'CS202002013_170'.
Invalid marker 'ENSMUSG00000028971' in file 'CS202002013_markers.tsv' with key 'CS202002013_170'.
Invalid marker 'ENSMUSG00000061762' in file 'CS202002013_markers.tsv' with key 'CS202002013_176'.
Invalid marker 'ENSMUSG00000028971' in file 'CS202002013_markers.tsv' with key 'CS202002013_176'.
Invalid marker 'ENSMUSG00000034324' in file 'CS202002013_markers.tsv' with key 'CS202002013_176'.
Invalid marker 'ENSMUSG00000010461' in file 'CS202002013_markers.tsv' with key 'CS202002013_177'.
Invalid marker 'ENSMUSG00000052353' in file 'CS202002013_markers.tsv' with key 'CS202002013_177'.
Invalid marker 'ENSMUSG00000020099' in file 'CS202002013_markers.tsv' with key 'CS202002013_178'.
Invalid marker 'ENSMUSG00000067028' in file 'CS202002013_markers.tsv' with key 'CS202002013_178'.
Invalid marker 'ENSMUSG00000098097' in file 'CS202002013_markers.tsv' with key 'CS202002013_184'.
Invalid marker 'ENSMUSG00000033308' in file 'CS202002013_markers.tsv' with key 'CS202002013_184'.
Invalid marker 'ENSMUSG00000059187' in file 'CS202002013_markers.tsv' with key 'CS202002013_184'.
Invalid marker 'ENSMUSG00000022790' in file 'CS202002013_markers.tsv' with key 'CS202002013_184'.
Invalid marker 'ENSMUSG00000036192' in file 'CS202002013_markers.tsv' with key 'CS202002013_188'.
Invalid marker 'ENSMUSG00000084890' in file 'CS202002013_markers.tsv' with key 'CS202002013_188'.
Invalid marker 'ENSMUSG00000034402' in file 'CS202002013_markers.tsv' with key 'CS202002013_188'.
Invalid marker 'ENSMUSG00000022419' in file 'CS202002013_markers.tsv' with key 'CS202002013_190'.
Invalid marker 'ENSMUSG00000059203' in file 'CS202002013_markers.tsv' with key 'CS202002013_190'.
Invalid marker 'ENSMUSG00000034009' in file 'CS202002013_markers.tsv' with key 'CS202002013_191'.
Invalid marker 'ENSMUSG00000045613' in file 'CS202002013_markers.tsv' with key 'CS202002013_191'.
Invalid marker 'ENSMUSG00000055214' in file 'CS202002013_markers.tsv' with key 'CS202002013_191'.
Invalid marker 'ENSMUSG00000020140' in file 'CS202002013_markers.tsv' with key 'CS202002013_195'.
Invalid marker 'ENSMUSG00000021541' in file 'CS202002013_markers.tsv' with key 'CS202002013_195'.
Invalid marker 'ENSMUSG00000098760' in file 'CS202002013_markers.tsv' with key 'CS202002013_195'.
Invalid marker 'ENSMUSG00000022112' in file 'CS202002013_markers.tsv' with key 'CS202002013_195'.
Invalid marker 'ENSMUSG00000111765' in file 'CS202002013_markers.tsv' with key 'CS202002013_201'.
Invalid marker 'ENSMUSG00000022231' in file 'CS202002013_markers.tsv' with key 'CS202002013_201'.
Invalid marker 'ENSMUSG00000039706' in file 'CS202002013_markers.tsv' with key 'CS202002013_201'.
Invalid marker 'ENSMUSG00000111765' in file 'CS202002013_markers.tsv' with key 'CS202002013_202'.
Invalid marker 'ENSMUSG00000025997' in file 'CS202002013_markers.tsv' with key 'CS202002013_202'.
Invalid marker 'ENSMUSG00000116029' in file 'CS202002013_markers.tsv' with key 'CS202002013_206'.
Invalid marker 'ENSMUSG00000110281' in file 'CS202002013_markers.tsv' with key 'CS202002013_206'.
Invalid marker 'ENSMUSG00000019935' in file 'CS202002013_markers.tsv' with key 'CS202002013_208'.
Invalid marker 'ENSMUSG00000028031' in file 'CS202002013_markers.tsv' with key 'CS202002013_208'.
Invalid marker 'ENSMUSG00000021318' in file 'CS202002013_markers.tsv' with key 'CS202002013_211'.
Invalid marker 'ENSMUSG00000036949' in file 'CS202002013_markers.tsv' with key 'CS202002013_211'.
Invalid marker 'ENSMUSG00000115529' in file 'CS202002013_markers.tsv' with key 'CS202002013_214'.
Invalid marker 'ENSMUSG00000013523' in file 'CS202002013_markers.tsv' with key 'CS202002013_214'.
Invalid marker 'ENSMUSG00000015202' in file 'CS202002013_markers.tsv' with key 'CS202002013_215'.
Invalid marker 'ENSMUSG00000032841' in file 'CS202002013_markers.tsv' with key 'CS202002013_217'.
Invalid marker 'ENSMUSG00000032517' in file 'CS202002013_markers.tsv' with key 'CS202002013_217'.
Invalid marker 'ENSMUSG00000032517' in file 'CS202002013_markers.tsv' with key 'CS202002013_218'.
Invalid marker 'ENSMUSG00000032702' in file 'CS202002013_markers.tsv' with key 'CS202002013_218'.
Invalid marker 'ENSMUSG00000064294' in file 'CS202002013_markers.tsv' with key 'CS202002013_224'.
Invalid marker 'ENSMUSG00000032796' in file 'CS202002013_markers.tsv' with key 'CS202002013_224'.
Invalid marker 'ENSMUSG00000048424' in file 'CS202002013_markers.tsv' with key 'CS202002013_225'.
Invalid marker 'ENSMUSG00000069378' in file 'CS202002013_markers.tsv' with key 'CS202002013_225'.
Invalid marker 'ENSMUSG00000010122' in file 'CS202002013_markers.tsv' with key 'CS202002013_227'.
Invalid marker 'ENSMUSG00000061171' in file 'CS202002013_markers.tsv' with key 'CS202002013_228'.
Invalid marker 'ENSMUSG00000039109' in file 'CS202002013_markers.tsv' with key 'CS202002013_231'.

[dosumis]

@BAevermann - We urgently need to fix this. Did you have time to take a look yet?

(Happy to have a short call to discuss if you'd like)

[dosumis]

File moved here to prevent pipeline failures: https://github.com/obophenotype/brain_data_standards_ontologies/blob/7f0bc284561004093ea62cb8883b90fc826250ab/src/markers/CS202002013_markers.tsv

[BAevermann]

Hey! So I just returned from a vacation where I did not have internet. Sorry about the delay. Can we have a call, perhaps tomorrow to discuss what need to be done?

b.

[BAevermann]

So I have a markers file with the duplicates removed. However, I was wondering If I should drop the clustername column as those seem to be changing (and will probably change in the future)?

let me know,

b

[hkir-dev]

We were using that column to check the compatibility of the dendrogram and the marker file. We expect clusterName to be same with the dendrogram node label. If this assumption is wrong, we can delete the column and I can update the related validation.

[BAevermann]

It just seems that the labels will be changing as the domain experts discuss what the data ultimately means. As such that column will be problematic and will require version tracking etc… To avoid that, we can just use the dendrogram node ID and let the community sus out the labelling… Maybe we should pitch this idea to David and Shawn, they might have an opinion. Cheers! b. From: Huseyin Kir ***@***.***> Date: Tuesday, September 21, 2021 at 9:46 AM To: obophenotype/brain_data_standards_ontologies ***@***.***> Cc: Aevermann, Brian ***@***.***>, Mention ***@***.***> Subject: Re: [obophenotype/brain_data_standards_ontologies] Fix broken marker file update (#103) We were using that column to check the compatibility of the dendrogram and the marker file. We expect clusterName to be same with the dendrogram node label. If this assumption is wrong, we can delete the column and I can update the related validation. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fobophenotype%2Fbrain_data_standards_ontologies%2Fissues%2F103%23issuecomment-924167355&data=04%7C01%7Cbaeverma%40jcvi.org%7Cb8807389daf54c6614b408d97d1f67e1%7C24d967f13ed84448baa6560ec572acb3%7C0%7C0%7C637678396111463340%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Utg02mrODnTG%2B50hgZCOue6S1o4i5EucK8gwXGZeJDI%3D&reserved=0>, or unsubscribe<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHUXLJQWUI63NSCAO2ZEJA3UDCZHRANCNFSM46Y67QZQ&data=04%7C01%7Cbaeverma%40jcvi.org%7Cb8807389daf54c6614b408d97d1f67e1%7C24d967f13ed84448baa6560ec572acb3%7C0%7C0%7C637678396111473337%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=jvcVxqQowYg3uyUj4ryv59wszvKW0nGv4VNmEKwVXEE%3D&reserved=0>. Triage notifications on the go with GitHub Mobile for iOS<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapps.apple.com%2Fapp%2Fapple-store%2Fid1477376905%3Fct%3Dnotification-email%26mt%3D8%26pt%3D524675&data=04%7C01%7Cbaeverma%40jcvi.org%7Cb8807389daf54c6614b408d97d1f67e1%7C24d967f13ed84448baa6560ec572acb3%7C0%7C0%7C637678396111483330%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2BFpu6mVS3JJ7%2F4FUGgKESZE%2Bif95zQuR9NPVhhfa9BA%3D&reserved=0> or Android<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fplay.google.com%2Fstore%2Fapps%2Fdetails%3Fid%3Dcom.github.android%26referrer%3Dutm_campaign%253Dnotification-email%2526utm_medium%253Demail%2526utm_source%253Dgithub&data=04%7C01%7Cbaeverma%40jcvi.org%7Cb8807389daf54c6614b408d97d1f67e1%7C24d967f13ed84448baa6560ec572acb3%7C0%7C0%7C637678396111483330%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Qeeqfs0STduImINRqobL9wzllbUMVm99cKlNmhJ%2Bkvc%3D&reserved=0>.

[shawntanzk]

If the processes are mostly automated and doesn't require manual curation (which I think in this case is true) I guess there isn't a need for the clustername (esp as what @BAevermann mentioned about names possibly changing which they already are, eg lamp5 -> lamp5-like). This label check would probably lead to a lot of failures that require fixes later down the road (I think if this gets integrated to the new build, it will already trigger a problem). So yeah, fully in agreement to remove it as long as all the processes are automated.

[BAevermann]

Ok. Uploaded without clustername, removed duplicates and continuing with ensemblIDs as thats what mouse used?