• Dupe Detection

    From nugax@21:1/167 to ALL on Thursday, October 13, 2022 09:07:31
    Hello All,
    So I need to write some duplication detection in echomail. My question is:
    Do you use MSG ID identifier, parse the actual msg body, or some other way
    to determine if a message is duplicated?

    Appreciate the help!

    -Nugax [CyberBBS]


    --- CyberBBS v1.0.8 2022/09/17 [LMDE 4/x86_64]
    * Origin: CyberBBS WHQ | http://www.cyberbbs.com (21:1/167)
  • From NuSkooler@21:1/121 to nugax on Thursday, October 13, 2022 10:33:22

    On Thursday, October 13th nugax was heard saying...
    Hello All, So I need to write some duplication detection in echomail. My question is: Do you use MSG ID identifier, parse the actual msg body, or some other way to determine if a message is duplicated?

    Yep, use MSGID if present. In enigma, imported messages also get a named UUID assigned (which must be unique) based on the following code:

    static createMessageUUID(areaTag, modTimestamp, subject, body) {
    assert(_.isString(areaTag));
    assert(_.isDate(modTimestamp) || moment.isMoment(modTimestamp));
    assert(_.isString(subject));
    assert(_.isString(body));

    if (!moment.isMoment(modTimestamp)) {
    modTimestamp = moment(modTimestamp);
    }

    areaTag = iconvEncode(areaTag.toUpperCase(), 'CP437');
    modTimestamp = iconvEncode(modTimestamp.format('DD MMM YY HH:mm:ss'), 'CP437');
    subject = iconvEncode(subject.toUpperCase().trim(), 'CP437');
    body = iconvEncode(
    body.replace(/\r\n|[\n\v\f\r\x85\u2028\u2029]/g, '').trim(),
    'CP437'
    );

    return uuidParse.unparse(
    createNamedUUID(
    ENIGMA_MESSAGE_UUID_NAMESPACE,
    Buffer.concat([areaTag, modTimestamp, subject, body])
    )
    );
    }


    --
    |08 ■ |12NuSkooler |06// |12Xibalba |08- |07"|06The place of fear|07"
    |08 ■ |03xibalba|08.|03l33t|08.|03codes |08(|0344510|08/|03telnet|08, |0344511|08/|03ssh|08)
    |08 ■ |03ENiGMA 1/2 WHQ |08| |03Phenom |08| |0367 |08| |03iMPURE |08| |03ACiDic
    --- ENiGMA 1/2 v0.0.13-beta (linux; x64; 16.16.0)
    * Origin: Xibalba -+- xibalba.l33t.codes:44510 (21:1/121)